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LESSONS LEARNED: HOW THE NEW 
ADMINISTRATION CAN ACHIEVE AN 
ACCURATE 

AND COST-EFFECTIVE 2010 CENSUS 


THURSDAY, MARCH 5, 2009 

U.S. Senate, 

Subcommittee on Federal Financial Management, 
Government Information, Federal Service, 

AND International Security, 

OF THE Committee on Homeland Security 
AND Governmental Affairs, 
Washington, DC. 

The Subcommittee met, pursuant to notice, at 2:31 p.m., in room 
SD-342, Dirksen Senate Office Building, Hon. Thomas R. Carper, 
Chairman of the Subcommittee, presiding. 

Present: Senators Carper, Burris, McCain, and Coburn. 

OPENING STATEMENT OF SENATOR CARPER 

Senator Carper. The Subcommittee will come to order. We wel- 
come you, one and all. I am delighted that Senator McCain is able 
to join us today, and to our guests, as well. 

The Subcommittee has responsibility for oversight in a number 
of areas. One of those is with respect to the Census Bureau’s prep- 
aration for the 2010 Census, and today we are going to hear from 
former Census Directors and experts within the statistical commu- 
nity who will offer their valuable insights into lessons learned from 
past Censuses. It is my hope that their experiences can help the 
Census Bureau conduct an accurate and cost-effective Census in 
2010. 

There is a well-known adage that knowledge is power and the 
Census is an important source of knowledge and information. Cen- 
sus data empowers citizens at every level of government and are 
integral in achieving equitable political representation and fair al- 
location of resources. Finding and enumerating nearly 300 million 
individuals in the correct location is, of course, an extremely 
daunting task. The 2000 Census involved the hiring of nearly half- 
a-million temporary workers, the opening of some 500 local Census 
offices nationwide, processing, I believe, 1.5 billion sheets of paper, 
and following up with 42 million non-responsive households. 

Given the sheer magnitude of such an undertaking, a short- 
coming in one area can quickly have a domino effect on other oper- 
ations. For example, a low mail response rate would increase the 

( 1 ) 
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non-response follow-up workload, which in turn would drive the 
Bureau’s staffing needs and drive up costs. 

With each Census, the challenge continues to grow in terms of 
cost and complexity as our population becomes larger, more di- 
verse, and increasingly difficult to enumerate. The cost of the 2010 
Census has escalated to an estimated $14 billion, and that is what 
my statement says. The cost of the 2010 Census has escalated to 
an estimated $14 billion, and I think that is true, making it the 
most expensive in the history of our country. Put another way, it 
will cost the Nation an estimated $100 or so to count each house- 
hold in 2010, compared with about $56 in 2013 dollars in 1970. The 
growing cost of the Census at a time when the Federal Government 
is facing unprecedented budget deficit highlights the importance of 
making sure that every additional dollar spent on the Census actu- 
ally improves the quality of the data. 

Although the 2000 Census was an improvement when compared 
to the 1990 Census, there were still many deficiencies. In 2000, 6.4 
million people were missed and 3.1 million people were counted 
twice, producing a net undercount of some 3.3 million people. 

I just interject, usually when we have an overcount, it is people 
that have more than one house, maybe a second home or a vacation 
home, or maybe they have a child who is in college in another 
State, and those are situations that lead to overcounts, and they 
usually occur among the more affluent families. On the other hand, 
the undercounts usually occur among a lot of minority families, 
whether African American or Latino or Native American. But we 
ended up with an undercount of about 6 million people, for the 
most part lower-income folks, and an overcount of about 3 million 
people the last time we did this, mostly of more affluent people. 
Neither one is a good situation, but that is not what we need for 
this Census and the conducting of this Census. 

At any rate, the 2010 Census is approaching rapidly, as we 
know, with the Census date less than 13 months away. The Bureau 
has faced many operational and organizational challenges that 
have jeopardized its success. These challenges include under- 
funding for outreach to minority communities and the colossal mis- 
management and failures of the contract for hand-held computers 
that led to an entire replan of the Census very late in the game. 
Senator Coburn and I have been working on this for several years 
under his leadership as the Chairman of this Subcommittee and 
more recently under my own. 

Further, I understand that the Bureau lacks plans for testing 
some of its key information technology systems. With such a sub- 
stantial reliance on new technology, a robust testing strategy is 
necessary to identify and correct any problems that may arise. 

I believe we are at a critical juncture. I don’t think it is over- 
stating things to say that the 2010 Census is approaching a state 
of emergency. Significant work still has to be done, and the Bureau 
does not have a Director in place to assist them in making these 
critical decisions. Last month, I sent a letter to President Obama 
urging him to nominate a new Director as soon as possible. I re- 
newed that request as recently as this morning. 



3 


It is my hope that we have learned from these valuable lessons 
and can continue to work together to ensure the success of the 
2010 Census. 

We look forward to the expert testimony here today from our dis- 
tinguished panel of witnesses. 

I would just say to our colleagues, and we have been joined by 
Senator Coburn, who knows these issues as well or better than 
me — Senator Coburn said that he didn’t think the people here 
knew who Senator McCain was so he is helping with his name tag. 
[Laughter.] 

But, Senator Coburn, we are in a situation where we have gone 
from an Administration where we didn’t have a Census Director in 
place for the longest time, we had to wait for a long time to get 
a nominee, finally got a nominee, a very good one, and he stayed 
with us for about a year and the beginning of this year he tendered 
his resignation letter along with a whole lot of other appointed offi- 
cials. So now we are waiting until we get a good solid replacement 
and time is wasting. 

Senator McCain, it is great to have you here. 

OPENING STATEMENT OF SENATOR MCCAIN 

Senator McCain. Thank you very much, Mr. Chairman. I want 
to thank you for holding this hearing and I want to thank the wit- 
nesses for appearing here today. I again thank you and Senator 
Coburn, who has a great sense of humor, I am sure you will note, 
for your attention to this issue. 

The first thing I would like to point out, I think it is pretty obvi- 
ous that we need to have a Director of the Census. I am sure our 
witnesses will be in total agreement on that aspect of this issue. 

We are nearing the time where certainly final preparations for 
one of the more important events are underway — I don’t think peo- 
ple appreciate the importance of the Census. It not only means a 
lot to us as far as Congressional districts are concerned, but so 
much of our Nation’s operations as far as apportionment of money 
for various programs, apportionment of responsibilities. So much of 
the things that we do, and, in fact, laws that we pass every day 
are implemented through the Census, guided by our knowledge, 
hopefully knowledge or lack of knowledge of the people we have in 
each State and each part of each State. 

So obviously a fair and accurate counting is critical. No State 
should be unfairly denied representation or funding for essential 
services because the Census Bureau can’t resolve problems that 
have plagued us for decades, and obviously one of those problems 
is undercounting. I guarantee you that whether it is totally accu- 
rate or not, there will be States that claim undercounting at the 
end of this process. I think our witnesses would agree with that. 

And I am one of them. My home State of Arizona suffered se- 
verely from undercounting in the 1990 Census. As a result, Arizona 
was denied an additional Congressional seat and lost millions of 
dollars in Federal revenue for schools, roads, housing, and other 
public services. 

Resolving the issue continues to spur debate and the need to 
make constant improvements to traditional enumeration methods 
remains a top priority. So the process must be fair. It has got to 
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be conducted in a manner that doesn’t discriminate and doesn’t dis- 
suade participation. 

I want to point out again that some recent activity on the part 
of the Census Bureau does not lend itself to increasing the con- 
fidence level. Of course, I am talking about the investment of mil- 
lions of dollars in hand-held computers that can’t deliver the capa- 
bility that was once promised. I know our Subcommittee examined 
the issue, but I still feel compelled to bring this up again because 
it is a terrible precedent to set and does not give us confidence. So 
it lost taxpayers’ money because of cost overruns and lost produc- 
tivity. We can’t afford, obviously, to waste that. 

I am concerned that we have enough time, and I will be inter- 
ested in hearing from our witnesses, about adequate testing of all 
critical systems and procedures before additional Census activities 
begin. I am most interested to hear from our GAO witnesses about 
the current status of these setbacks and how much delay this mis- 
management has caused. 

I want to thank you, Mr. Chairman. I want to thank our wit- 
nesses. Again, this is a very important process that this Nation is 
about to embark on. You mentioned people that have second 
homes. I also think that this is a period of great mobility in Amer- 
ica from one place to another for economic reasons and others, in- 
cluding the traditional mobility of Americans. I do not know, 
maybe one of our witnesses knows how many people move from one 
State to another in the course of a year, but it is significant and 
on the increase. 

We have an obligation to ensure that every American is counted 
and counted accurately and I thank you, Mr. Chairman, and I want 
to thank Senator Coburn for his involvement for many years in this 
issue. Thank you. 

Senator Carper. Thank you for your statement and thank you 
for joining us in this effort. 

I am going to go to Senator Burris and then to Senator Coburn. 

OPENING STATEMENT OF SENATOR BURRIS 

Senator Burris. Thank you very much, Mr. Chairman. 

Senator Carper. Welcome. We are delighted that you are here. 

Senator Burris. My pleasure. I would like to extend my warm 
welcome to our distinguished valued panelists. I am so pleased to 
see that you are carrying out your constitutional and committed re- 
sponsibilities to inform us as officials. 

As we approach what projections indicate will be the most expen- 
sive Census in history, we must assure that sound leadership aids 
its execution. It is crucial that we take action quickly to guarantee 
success. Constituents must trust us to spend their money wisely 
and we must ensure that we plan both comprehensive and respon- 
sible. 

America has changed greatly in the last 10 years and I fear that 
some citizens may be less likely to participate in the Census. We 
already face significant language barriers and we now must at- 
tempt to overcome suspicion of legitimacy. With identity theft on 
the rise, many ordinary Americans may overlook the necessity of 
the Census for fear of their personal safety, so we must be very 
concerned about what is happening in the minds of our citizens. 
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Fortunately, through the knowledge of the previous experience, 
hard work, and development of new technology, significant opportu- 
nities now exist for us to improve the process, and I am with Sen- 
ator McCain on his comments. 

The undercount, we hear so much about the undercount. I live 
in Chicago and in a metropolitan area, we are always complaining 
about the fact that in those areas, we are not counted because no- 
body wants to go up into especially what we call the developments. 
Some people refer to them as projects. A lot of them have been torn 
down in Chicago now, but we hope to find where all those people 
were relocated and try to identify those people because that is 
where a lot of the allocations are determined. 

I will have some questions later, and Mr. Chairman, I might 
have to run out, but I will come back with some questions later on. 

Thank you, Mr. Chairman. 

Senator Carper. Senator Burris, thank you very much. 

I am delighted again that my partner in this initiative and part 
of my good oversight is here with us. Senator Coburn. Please pro- 
ceed. 


OPENING STATEMENT OF SENATOR COBURN 

Senator Coburn. I will just spend a short amount of time. The 
best recommendation I could make to the President is to bring Mr. 
Murdock back. He did a great job while he was there. I think it 
is unfortunate for us as a Nation that he left in the midst of 
straightening out a lot of the problems that were there. I will save 
all the rest of my comments for the time of which we have ques- 
tioning. 

Senator Carper. All right. Thanks very much. 

Let me briefly introduce our witnesses, and we will start with 
the Hon. Barbara Bryant. Welcome. She was Director of the Cen- 
sus Bureau from 1983 to 1991, and during her tenure, she directed 
the 1990 Decennial Census of Population and Housing, the 1992 
Economic and Agricultural Census, and other major surveys. I be- 
lieve you were appointed by President George Herbert Walker 
Bush and confirmed by the Senate as the first woman to head the 
Census Bureau in 200 years of Census taking. Dr. Bryant received 
her Bachelor’s degree from Cornell University and her M.A. and 
Ph.D. from Michigan State University — a Spartan. Welcome today. 

John Thompson, I call him the real John Thompson, is the Presi- 
dent of the National Opinion Research Council at the University of 
Chicago. Mr. Thompson came to the Council after a 27-year career 
at the Census Bureau, where as one of the Bureau’s most senior 
career officers he had the responsibility for all aspects of the 2000 
Census, including management, operations, and methodology. Mr. 
Thompson attended Virginia Tech University, where he was award- 
ed a B.S. and a Master’s of Science degree in mathematics. 

Next, Robert Goldenkoff, Director of Strategic Issues at GAO, 
where he is responsible for reviewing the 2010 Census and govern- 
ment-wide human capital reforms. Mr. Goldenkoff has also per- 
formed research on issues involving transportation security, human 
trafficking, and Federal statistical programs. He received his Bach- 
elor’s in political science and Master’s in public policy from George 
Washington University. 
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Dave Powner, good to see you again. Thank you for joining us. 
He has over 20 years of experience in information technology issues 
in both the public and private sector. He is currently responsible 
for a large segment of GAO’s information technology work, includ- 
ing systems development, IT investment and management, health 
IT, and cyber critical infrastructure protection reviews. He is no 
stranger to this Subcommittee. Thank you for joining us again 
today. 

Lawrence D. Brown, Professor in the Department of Statistics at 
the Wharton School at the University of Pennsylvania. He is a 
member of the National Academy of Sciences and has served on 
several committees and panels of the National Research Council, 
including the Committee on National Statistics. Dr. Brown’s work 
includes evaluations on the design and methodology of both the 
2000 and the 2010 Census programs. He received a Bachelor of 
Science degree in mathematics from the California Institute of 
Technology and a Ph.D. in math statistics from Cornell. 

And last but not least, Robert Hill, a sociologist who recently re- 
tired as Senior Researcher at Westat, a research firm in Rockville, 
Maryland. He was Chair of the U.S. Census Bureau Advisory Com- 
mittee on the African American Population for both the 1980 and, 
I think, the 2000 Censuses. Dr. Hill received his Bachelor’s of Art 
in sociology from the City College of New York and a Doctorate in 
sociology from Columbia University. 

We would have one other person here with us today, Vincent 
Barabba, former Census Director during the Nixon and Carter ad- 
ministrations. He was scheduled to participate in our hearing 
today. Due to a series of unanticipated events, he is unable to join 
us. His testimony will be submitted for the record and the Sub- 
committee looks forward to working with him in the future as we 
continue our oversight of the 2010 Census.^ 

With those introductions behind us, let me just say, Ms. Bryant, 
we welcome you here. We are delighted that you are going to be 
our lead-off hitter. We will go right down the line and then we will 
ask questions. Thank you. 

TESTIMONY OF BARBARA EVERITT BRYANT, PH.D., 2 FORMER 
DIRECTOR, U.S. CENSUS BUREAU 

Ms. Bryant. Thank you. Chairman Carper and Acting Ranking 
Member McCain. You have been given my introduction statement. 

Lesson one for the 2010 Census is the one that all of you have 
referred to, and that is the importance of getting a new Census Di- 
rector in immediately. I am pleased to see you are pressing on the 
President to make the nomination. On this, I really speak from ex- 
perience, because 

Senator Carper. If I could interrupt, I also pressed this morning 
on the President’s nominee for Commerce. I ran into Governor 
Locke yesterday and again today and I said, if you haven’t started 
thinking about who you would like to have on the Census Bureau, 
start thinking about it right now. 

Ms. Bryant. Thank you very much 


^The prepared statement of Mr. Barabba appears in the Appendix on page 33. 
2 The prepared statement of Ms. Bryant appears in the Appendix on page 38. 



7 


Senator Carper. I brought it to his attention. 

Ms. Bryant [continuing]. From all of us. 

Twenty years ago, I became Director in the same election cycle 
we are in now, that is in the Presidential election in the year end- 
ing in eight and nomination in the year ending in nine. However, 
I was not in office until December 7, 1989, 3 weeks before the start 
of the Census year. I was eventually confirmed by the Senate. 

Would Census procedures have been different if I had been in of- 
fice sooner? Definitely, and I elaborate on this in my written testi- 
mony. 

I am a supporter of making the job of Director of the Census Bu- 
reau a 5-year Presidential appointment, starting in the years one 
and six. The planning cycle for operations as large as the Decennial 
and Economic Censuses are long and only with a several-year lead 
time can a Director have any real input into what is going to be 
done. I also will say it is no fun for a Census Director to sit in front 
of Congressional committees like this defending operations in 
which he or she had no input. 

Lesson two is that a major professional coordinated communica- 
tions and advertising campaign is vital to Census success and accu- 
racy. Such a campaign requires a major financial outlay. It has two 
components, a large volume of inexpensive promotional materials 
that can be handed out or posted at the local level; and radio, TV, 
and newspaper spots professionally produced with goals of reaching 
both the mass national and targeted audiences. 

The advertising campaign has got to be on a scale comparable to 
what a private sector firm would use to introduce a new product. 
After all, the Census is a new product to everyone in their 20s, and 
is a 10-year-old, half-forgotten product for anyone 30 and over. 

Lesson three is outreach to hard-to-count segments of the popu- 
lation. Through partnerships with geographic, ethnic, and racial or- 
ganizations, we can help reduce the undercount. The Census his- 
torically has fully counted some segments of the population, par- 
ticularly homeowners and older Americans. It falls short of fully 
counting the very mobile, the renters, the young people, and par- 
ticularly those in Hispanic, Latino, African American, and Amer- 
ican Indian communities. These hard-to-count are best reached 
with one-on-one contacts from local people and organizations in 
whom they have trust. 

Communicating the fact that the Census Bureau will not give in- 
formation from their Census forms to any other organization or in- 
dividual is a very hard message to get across. Only trusted sources 
can convince the reluctant, the fearful, or the uninformed that the 
Census Bureau does not give information to the INS, the IRS, land- 
lords, ex-spouses, or mothers-in-law. [Laughter.] 

In addition to implementing these three lessons, three other fac- 
tors will help improve the accuracy in 2010. One is the American 
Community Survey, which is now ongoing, and it replaces the in- 
formation formerly gathered on the long form with about 50 ques- 
tions that went to 17 percent of households. The long form always 
had a several percent lower mail return than the short form. 

The second thing that is going to help is the downturn in employ- 
ment. This is a national tragedy, but it does have the benefit for 
the Census Bureau that they are going to have a bigger pool of peo- 
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pie from which to choose their temporary employees. My Census, 
we had 5 percent unemployment and the pool had dried up to be 
a puddle. 

And third, the undercount research done after all the recent Cen- 
suses identifies very precisely where non-respondents are geo- 
graphically so they can be targeted. Such research is important to 
fund for every Census. 

But now the big inhibitor to a good count in 2010 is the fear in 
the Hispanic-Latino communities. The current Immigration and 
Naturalization raids on such communities, on employers and neigh- 
borhoods is bound to depress cooperation. I flew in yesterday from 
Phoenix, Senator McCain, where I do have a second home and 
some households there include both legal and undocumented immi- 
grants. You can imagine when a Census taker goes into those 
neighborhoods and says, “I am from the Census Bureau and I have 
a few questions from the Government,” what a warm reception 
they may receive. 

Finally, you asked us to comment on a cost-effective Census. 
Well, counting every person and household is never going to be a 
cheap operation, particularly with all this follow-up on the hard-to- 
count. But with this experience in the logistics of the operation and 
its magnitude, and I will add the wonderful employees, the long- 
term career employees at the Census Bureau, the Census Bureau 
probably does as cost-effective a job as any organization could. But 
current staff, not the Director of a $2.6 billion Census conducted 
20 years ago must report to you on present efforts to be cost effec- 
tive. 

Senator Carper. Thank you very much. Mr. Thompson. 

TESTIMONY OF JOHN THOMPSON, i PRESIDENT, NATIONAL 
OPINION RESEARCH COUNCIL 

Mr. Thompson. Thank you, Mr. Chairman and Members of the 
Subcommittee. I am truly honored to be glad to speak to you about 
the 2010 Census. As you asked, I will talk about some of the suc- 
cesses in 2000 and relate them to the risks with respect to the 2010 
Census. 

Before I start, I would quickly like to recognize the fact that I 
worked with a lot of the people at the Census Bureau right now. 
I know that they are motivated to do high-quality work. They are 
nonpartisan and they are very good and my remarks are intended 
to help them with their effort, not in any way to criticize them. 

So starting with Census 2000, I think the first factor I would 
mention in success, and you will hear a lot of the same things, I 
think, is the unprecedented support that the 2000 Census received. 
One example of that was that the mail response rate was 67 per- 
cent, which was higher than the 1990 Census rate of 65 percent. 
That was the first Census where the decline in mail response rate 
had been reversed. 

I think there were three factors that contributed to that. The 
first is a paid advertising campaign. It was the first Census that 
used a paid advertising campaign. 


^The prepared statement of Mr. Thompson appears in the Appendix on page 44. 
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The second was a very well-funded effort to establish partner- 
ships with local community groups. 

And the third was a very effective communications strategy that 
reached out to numerous stakeholders, including the Congress, 
State, local, and Tribal governments, and a variety of advocacy 
groups. 

In addition, the non-response follow-up operation, the most dif- 
ficult part of the Census, was finished in 9 weeks. One of the fac- 
tors that contributed to that was, in my opinion, the fact that the 
public was highly motivated to cooperate and we didn’t have to 
visit the households too many times. There were other factors 
which I will mention, too. 

Another factor in the success of the Census was that we had a 
Director in place in time to provide leadership and guidance. That 
Director was Kenneth Pruitt. I had the privilege of working with 
him. He provided leadership, guidance, and set an environment up 
where the career people could be successful. I also would note that 
I had the pleasure of working with Dr. Bryant on the 1990 Census 
and that was also a fine experience. 

Another factor was that we had a very strong management team 
in place that was very experienced and included managers with 
both Census experience and managers from outside of the Census 
Bureau that brought different perspectives to problem solving, and 
we had the team in place in time to make some significant con- 
tributions. 

Another factor was that the Census 2000 field effort was very 
well funded and well managed. We had done studies that linked 
pay for retention and we had an excellent group of regional direc- 
tors who managed the Census. We were able to recruit and retain 
a workforce to do the job. 

We also had effective usage of private sector contractors in 2000 
to provide advance technology solutions to our data capture oper- 
ations. We had private sector contractors that ran some very large 
facilities, recruited a lot of staff, and put in place optical scanning 
and intelligent character recognition software that allowed us to 
capture over 80 percent of the handwritten entries on the Census 
forms with a very high degree of accuracy. 

The final factor I will mention which is relevant to this Census 
is that we had a thorough testing of all of our operational systems. 
We had a dress rehearsal in 1998 where we tested our systems 
from start to finish. That was very important. We also, I will note, 
had to change the Census design. There was a controversy over 
Census. We were going to add two tracts, and in 1999, we de- 
cided — the Supreme Court decided that we would not use sampling 
for the count, so we had a redesigned Census that did not use sam- 
pling. The fact that we had our systems tested, we were able to 
modify them and move forward. 

So for 2010, the major risk is in systems development and test- 
ing. I think the Subcommittee said that and it is fairly obvious. 
They had to abandon their plan to use hand-helds. They are going 
back to a paper-based system. They have taken over the control 
system from a private contractor. I think the best recommendation 
I can make is that they need to do a full-scale, large field test of 
that system to conduct a response follow-up, to collect some data. 



10 


to make sure that they have interfaces with all the key systems, 
and to make sure that interviewers, Census enumerators, fairly in- 
experienced people can utilize these systems. 

I will say just a couple more things. We have talked about the 
Director. I agree with that. 

Senator Carper. When you say agree with it, be more specific in 
what you agree with, the importance of getting a good one or 

Mr. Thompson. I agree that we need a Director of the Census 
Bureau as soon as possible. Like I said, having a Director in place 
during the Census is just incredibly important, and in the period 
preceding the Census. 

They need also to establish a communications lead. That needs 
to be appointed. They don’t have one right now. 

The management staff, I think they are doing a good job. They 
are working as hard as they can. I think they are very thin. I think 
they should reach out to some of the other Federal agencies to look 
for some more talent. 

Their coverage measurement system is currently scheduled in a 
fashion where I am concerned that it won’t produce accurate meas- 
ures. It is scheduled to take place too far after Census Day and I 
think there will be issues with recall bias associated with it. They 
need to tighten the schedule up similar to previous Censuses. 

And finally. Dr. Brown will talk a lot, I think, about the experi- 
mental program for the 2020 Census, but I encourage the Sub- 
committee to challenge the Census Bureau to develop plans for a 
different type of Census for 2020, one that is less expensive and re- 
lies on alternative methods of data collection. 

That concludes my remarks. Thank you. 

Senator Carper. Thank you, Mr. Thompson. Mr. Goldenkoff. 

TESTIMONY OF ROBERT GOLDENKOFF, i DIRECTOR, STRA- 
TEGIC ISSUES, U.S. GOVERNMENT ACCOUNTABILITY OFFICE 

Mr. Goldenkoff. Chairman Carper, Ranking Member McCain, 
Members of the Subcommittee, thank you for the opportunity to be 
here today to provide a progress report on the 2010 Census. I am 
here with Dave Powner, a Director on GAO’s Information Tech- 
nology team. 

As requested, in our remarks today, I will provide a broad over- 
view of the status of key Census-taking operations and Mr. Powner 
will focus on the findings and recommendations contained in our 
report on IT testing which we are releasing today. 

This afternoon’s hearing is particularly timely. It was exactly one 
year ago today that GAO designated the 2010 Census as a high- 
risk area for three reasons. First, there were weaknesses in the 
Census Bureau’s IT acquisition and contract management function. 
Second, there were problems with the performance of hand-held 
computers used to collect data. And third, the ultimate cost of the 
Census is uncertain, although it is currently estimated at more 
than $14 billion. 

At the same time, just over one year from now, it will be Census 
Day. Little time remains to address the challenges that have 
emerged thus far and make final preparations for the numerous op- 


^The prepared statement of Mr. Goldenkoff appears in the Appendix on page 49. 
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erations that will take place throughout 2010. In short, today’s 
hearing is a convenient waystation on the road to Census Day, a 
time to look back on the Census Bureau’s efforts over the past year 
to address the operational challenges that have emerged thus far 
as well as to look ahead to what the Bureau needs to do in the 
coming months to help ensure a successful head count. 

Importantly, the Bureau has made commendable progress over 
the past year in rolling out key components of the Census and to 
strengthen certain risk management efforts. Still, the Census re- 
mains high-risk because a dress rehearsal of all Census operations 
that was planned for 2008 was curtailed. As a result, critical activi- 
ties, including some that will be used for the first time in a Census, 
were not tested in concert with one another or under Census-like 
conditions. 

The bottom line is that key Census-taking activities, including 
those that will ultimately drive the final cost and accuracy of the 
count, continue to face challenges and the Bureau’s overall readi- 
ness for 2010 is uncertain. 

One such challenge is building the Bureau’s address list. Because 
a complete and accurate address list is the cornerstone of a suc- 
cessful Census, the Bureau has a number of operations aimed at 
including every residence in the country and works with the U.S. 
Postal Service, agencies at all levels of government, as well as a 
number of non-governmental entities. In a few weeks, the Bureau 
will send thousands of workers to walk every street in the country 
to update the Census address list and maps in an operation called 
address canvassing. Census workers will use hand-held computers 
to collect data. 

As you know, when the devices were tested, they experienced 
performance problems, such as freeze-ups and unreliable trans- 
missions. The Bureau took steps to fix these issues and the results 
of a small-scale test held last December are encouraging. Nonethe- 
less, more information is needed to determine the Bureau’s overall 
readiness for address canvassing, as a field test was not an end- 
to-end systems evaluation, did not validate training, help desk sup- 
port, and other requirements, and did not include urban areas. 

Uncertainties also surround the Bureau’s ability to implement 
operations that will be used for the first time in a decennial Cen- 
sus, including the targeted second mailing to reduce the non-re- 
sponse follow-up workload and the need to fingerprint temporary 
Census workers. The Bureau’s readiness for these activities is un- 
certain because they have not been tested under Census-like condi- 
tions. 

Another challenge facing the Bureau, as we have mentioned, is 
reducing the undercount. As with past enumerations, the Bureau 
is putting forth tremendous effort to reach groups that are often 
missed by the Census, such as minorities, renters, and people with 
limited English proficiency. For example, the Bureau plans to pro- 
vide language assistance guides in 59 languages, an increase from 
49 languages in 2000. 

Although the effects of the Bureau’s communication efforts are 
difficult to measure, the Bureau reported some positive results 
from its 2000 marketing efforts with respect to raising awareness 
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of the Census. Still, a longstanding challenge for the Bureau is con- 
verting awareness of the Census into an actual response. 

Some specific hurdles that need to be overcome include the Na- 
tion’s linguistic diversity and privacy concerns and a post-Sep- 
tember 11, 2001 environment that could heighten some groups’ 
fears of government agencies. 

In summary, just 13 months remain until Census Day. At a time 
when major testing should be completed and there should be con- 
fidence in the functionality of key operations, the Bureau instead 
finds itself managing late design changes and developing testing 
plans. The Bureau has taken some important steps towards miti- 
gating some of the challenges that it has faced to date, yet much 
remains uncertain and the risks to a successful decennial Census 
continue. 

TESTIMONY OF DAVID POWNER,i DIRECTOR, INFORMATION 

TECHNOLOGY MANAGEMENT ISSUES, U.S. GOVERNMENT AC- 
COUNTABILITY OFFICE 

Mr. PowNER. Chairman Carper, Ranking Member McCain, and 
Members of the Subcommittee, the accuracy of the 2010 Census de- 
pends in large part on the proper functioning of IT systems both 
individually and when integrated together. 

Mr. Chairman and Dr. Coburn, your oversight of the Bureau’s ac- 
quisition of IT systems was critical last year. In particular, the 
field data collection system is no longer spiraling out of control and 
that contract is $500 million less than the initial estimates pro- 
vided at your hearings last summer. Your oversight is needed once 
again in the technology area to ensure that between now and Cen- 
sus Day, these systems are rigorously tested. 

Today, we are releasing our latest report completed at your re- 
quest which highlights that significant testing remains. Six major 
systems need to complete system testing and much integration 
testing needs to occur. Plans for conducting this testing are not 
completely in place. In order to ensure effective test execution, the 
Bureau needs comprehensive metrics to monitor test completion 
and effective executive-level oversight to keep the pressure on and 
to manage risks. 

Our report contains 10 detailed recommendations that the Bu- 
reau has agreed to address. For example, integration testing in- 
cludes testing the interfaces or the handshake between systems. 
Our work found that not only are there not complete plans for inte- 
gration testing of these interfaces, but there is not even a master 
list of interfaces. Not having such basic information at this stage 
is unacceptable and our recommendations call for the Bureau to, 
one, develop a master list of interfaces; two, prioritize the inter- 
faces based on criticality and need date; and three, to use this in- 
formation to develop all the needed integration test plans. 

To the Bureau’s credit, we are seeing more plans and better 
metrics, but there is still much work ahead in both areas. I would 
like to stress the need to prioritize. It is likely that the Bureau will 
not have enough time to test everything. Testing the most impor- 


^The prepared statement of Mr. Powner appears in the Appendix on page 68. 
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tant aspects of certain systems, interfaces, and operations is critical 
given the limited time remaining. 

Mr. Chairman, again, thank you for your leadership and I will 
look forward to your questions. 

Senator Carper. Thank you very much, Mr. Powner, for you and 
others of your colleagues at GAO for helping Senator Cohurn and 
I and our staffs in this effort. Thank you. 

Dr. Brown, please. 

TESTIMONY OF LAWRENCE D. BROWN, PH.D.,i CHAIR, COM- 
MITTEE ON NATIONAL STATISTICS, NATIONAL ACADEMY OF 

SCIENCES 

Mr. Brown. Good afternoon, Mr. Chairman. Thank you for invit- 
ing me to talk to you this afternoon. As you noted, I am a professor 
of statistics at the Wharton School of Business and I have been ac- 
tively interested in issues relating to the Decennial Census for over 
a decade. 

Among other things, I have served on several other National 
Academy of Sciences advisory panels involving Census issues, and 
currently I am Chair of a panel to review the Census program of 
evaluations and experiments. Many of my comments this afternoon 
are drawn from a very recent letter report of this panel that was 
mailed to Thomas Mesenhourg as Acting Director of the Census 
Bureau. 2 

There are three issues I would like to bring to your attention 
from our panel’s reports. Two of these involve research and plan- 
ning that should be part of the 2010 Census and the third concern 
is a more immediate one about research that should be conducted 
before fielding the 2010 Census, and I will start with that issue. 

Actually, I am the third person on this panel, as well as you, to 
have talked about the concern with the operating control system. 
As you have remarked and several others have remarked, the 
hand-held devices that were scheduled to be used in the non-re- 
sponse follow-up portion of the Census, academically termed 
NRFU, were withdrawn from use and that led to a considerable in- 
crease in Census costs. 

But I want to focus on a different aspect of this forced change 
and it is really the aspect that both John Thompson and Robert 
Goldenkoff have also mentioned. These devices had been des- 
ignated to form the core of the operating control system for NRFU. 
NRFU, as you noted, is a process that has over half-a-million peo- 
ple in the field operating out of many local offices. This army of 
people requires a system to keep track of it and the Census Bureau 
is now in the process of restructuring their entire operating control 
system because of the necessity of removing the hand-helds. 

So because of the timing of the decision to revert from hand-held 
computers to paper-based NRFU, the 2008 dress rehearsal did not 
test NRFU at all, and this was, of course, a major gap in Census 
testing. Because it wasn’t tested, the dress rehearsal provided no 
information on interaction of NRFU processes with the redesigned 


^The prepared statement of Mr. Brown appears in the Appendix on page 78. 
2 The letter from Mr. Brown appears in the Appendix on page 82. 
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coverage follow-up operation and various other components of the 
process. 

As a remedy, the Census Bureau has scheduled a number of iso- 
lated component tests, but this testing, component-wise testing 
strategy puts the Bureau in an extremely risky position. So I want 
to just reemphasize that the Bureau needs to perform as full and 
realistic an operational test from start to finish of this system as 
they can, including all of the interactions among the various com- 
ponents. 

So the two research issues that I want to mention, and I will try 
and be brief in mentioning them, the first of these involves admin- 
istrative records that could be used in the Census. The Census Bu- 
reau in the past two decades has conducted a research program to 
see whether administrative records could be used to increase the 
accuracy and reduce the cost of the Census. We believe that they 
offer the best chance of accomplishing those ends, but there are not 
scheduled to be any major tests of administrative records in the 
2010 Census, and given that their use provides one of the few op- 
portunities to substantially reduce Census field costs, we believe 
that the Census Bureau should devote serious effort and attention 
to including an experiment or research during the 2010 Census to 
see whether such records can be used in the future. 

And finally, with respect to the Internet, the Internet is another 
opportunity for cost reduction and improvement in data quality and 
the Census Bureau has no plans to incorporate Internet question- 
naires in 2010 or to perform research on how that would enable 
them to be used in 2020. So we believe that, if for no other reason 
than to avoid looking out of step with modern data collection and 
because of their problems, the Bureau should be conducting some 
coordinated program of Internet research using Internet devices. 

I think I will close here. Thank you for the invitation to testify, 
and I would be happy to address any further questions. 

Senator Carper. Great. Thank you. Thank you. Dr. Brown. Dr. 
Hill, please. 

TESTIMONY OF ROBERT B. HILL, PH.D.,i SOCIOLOGIST AND 

FORMER CHAIR OF THE ADVISORY COMMITTEE ON THE AF- 
RICAN AMERICAN POPULATION, U.S. CENSUS BUREAU 

Mr. Hill. Good afternoon, Mr. Chairman and Members of this 
Subcommittee. I am pleased to be invited to provide testimony on 
this very important subject. 

My testimony will focus on a major lesson learned from prior 
Censuses: The importance of developing strong partnerships and 
community outreach strategies with hard-to-count populations in 
order to reduce the minority undercount in the Census. 

My initial experience with Decennial Censuses goes back to 1969, 
when I was appointed National Director of the National Urban 
League’s 1970 Census Project. This project was launched by Whit- 
ney Young, who was the Executive Director of the National Urban 
League at that time, and it was designed to reduce the black 
undercount in the Census. This was the first national partnership 
between the U.S. Census Bureau and a minority organization with 


^The prepared statement of Mr. Hill appears in the Appendix on page 95. 
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over 100 branches throughout the country. The primary purpose of 
the 1970 Census Project, whose slogan was, “Make Black Count,” 
was to educate African Americans about the importance of the Cen- 
sus and to encourage them to cooperate. 

Our community outreach project was successful in convincing 
large segments of the African American community to participate 
in the 1970 Census. However, post-Census studies revealed that 
there was still a sizeable undercount of African Americans and 
other minorities in the 1970 Census. We believe that a major rea- 
son for the historic undercount of minority groups was the failure 
of the Census Bureau to adequately involve minority representa- 
tives in the advance planning and implementation of Decennial 
Censuses. 

Therefore, in his testimony to the House Census Oversight Com- 
mittee in September 1970, Whitney Young recommended that the 
Census Bureau establish ongoing minority advisory committees to 
improve its strategies for reducing the undercount. Indeed, in 1975 
under the visionary leadership of Vincent Barabba as the Census 
Bureau Director, the first Race and Ethnic Advisory Committees, 
which are also called REACs, were formed to assist the Bureau in 
planning for the 1980 Census. The initial REACs comprised four 
minority groups: African Americans, American Indians, Hispanics, 
and Asians. For the 2000 Census, a fifth group was added. Native 
Hawaiians and Pacific Islanders. 

I served as Chair of the African American Committee in the 
planning for the 1980 Census, was reappointed for the 2000 Cen- 
sus, and was involved in the initial planning for the 2010 Census. 
Over the years, the Bureau has improved its methods for enumer- 
ating the African American population and for reducing the 
undercount among minorities. While there is still a differential 
undercount, its size has steadily declined. 

For example, while the Bureau estimated that it missed about 
1.5 million, or 8 percent, of the black population in 1970, it failed 
to count about 1 million, or 1.8 percent of them, in the 2000 Cen- 
sus. But the group with the highest undercount rates in Decennial 
Census, regardless of race or ethnicity, are children under 18. 

One of the most effective strategies the Bureau has used to re- 
duce the minority undercount is to develop strong partnerships 
with minority groups in all phases of Census planning and to con- 
duct aggressive education and outreach campaigns in hard-to-count 
communities. Based on my experience with prior Censuses, I would 
like to offer some recommendations. 

First, I think it is very important that Congress provides the 
Census Bureau with adequate resources to undertake the mam- 
moth task of achieving a fair and accurate count. President Obama 
and Members of Congress should be congratulated for including an 
additional $1 billion in the President’s stimulus bill to enhance the 
Bureau’s enumeration activities in 2010. I was especially pleased 
that the bill stipulates that the Bureau can spend up to $250 mil- 
lion for its partnership program and outreach efforts to minority 
communities and hard-to-reach populations. 

Second, because of its comprehensive scope, the 2010 Census will 
directly stimulate this economy by hiring over half a million Cen- 
sus takers across the Nation. It is essential that there is an eth- 
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nically and racially diverse workforce, from the staff in the district 
offices to the enumerators in the neighborhoods and barrios. Mem- 
bers of hard-to-count populations should be adequately represented 
at all levels of Census hires, especially among the new partnership 
specialists. 

Third, one of the remarkable successes of the 2000 Census was 
the use of paid advertising to communicate messages about the im- 
portance of the Census to all groups. The fact that minority-owned 
advertising firms were extensively used to reach their respective 
groups played a large part in reducing the undercount in minority 
communities in the 2000 Census. 

Fourth and finally, the Bureau should permit members of its five 
Race and Ethnic Advisory Committees to play a more prominent 
role in implementing the 2010 Census, such as recommending part- 
nership specialists and minority advertising firms, distributing for- 
eign language Census forms, and identifying local sites for training 
Census workers and for serving as assistance centers to aid the el- 
derly and other individuals to fill out their forms. 

These are a few suggestions I have to offer to ensure that the 
2010 Census will be one of the most accurate and equitable enu- 
merations in our history. Thank you for this opportunity. 

Senator Carper. Dr. Hill, those were great recommendations. 

I was asked in a media interview earlier today why we are hav- 
ing this hearing. One of the things, we have a responsibility to do 
is oversight. Dr. Coburn has tried very hard, both as the Chairman 
of the Subcommittee and as Ranking Member of this Subcommittee 
joined by me, to ensure we meet our responsibility for oversight. I 
replied to the reporter who asked me the question, “What do you 
hope to accomplish from your hearing today,” and what I hope to 
accomplish is, one, I would like to ask each of you to send me two 
names by close of business tomorrow of somebody that you think 
would be an excellent Director of the Bureau of the Census. By 
close of business tomorrow, give us two names of people you think 
are well equipped to do this job. I hope the Administration has 
somebody that they are vetting, that they are close to submitting, 
but just in case they don’t, I want to make sure that we can give 
them a bigger talent pool to draw from. 

At least one of you said in your comments earlier today, talking 
about sort of a silver lining, high unemployment times right now. 
Well, the silver lining in that is that there is a great pool of talent 
from which to draw to work in the Census, whether enumerators 
or others, that will help reduce, I think, the mistakes that are 
made as we count people. So that is a potential for something good 
happening. 

But give us a couple of good names, each one of you, if you 
would, by close of business tomorrow. Thank you very much. 

The other thing I mentioned in response to the reporter’s ques- 
tion today, I said I want to make sure that when that new Director 
of the Bureau of the Census is identified, vetted, nominated, con- 
firmed, goes to work, that he or she have a pretty good to-do list 
that we have provided to him or her from some people who have 
been there and done this, not just once, in some cases twice and 
three times. We want to make sure that this Subcommittee is bet- 
ter equipped to do our job for oversight. 
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Dr. Hill was good enough to give us four recommendations. 
Would you just run through those again real quickly, and then 
what I am going to do is ask the panel to react to those rec- 
ommendations very briefly and to say if there are some that you 
would like to add to that. I think you have in your individual testi- 
mony. But beyond making sure we get an excellent Director on 
board soon, in place, just give us those four recommendations 
again. Dr. Hill. 

Mr. Hill. Essentially, the first was that the Bureau has ade- 
quate financial resources to conduct the Census. 

Senator Carper. And I think you said we seem to have done 
that. 

Mr. Hill. That is right. 

Senator Carper. Good. 

Mr. Hill. Second is that they should hire a workforce that is eth- 
nically and racially diverse and represents the hard-to-count and 
minority community populations. 

The third was that we should use the paid advertising as was 
done in 2000, but also use minority-owned firms that can effec- 
tively target their messages to various hard-to-reach groups. 

And the fourth was to permit the members of the REAC Commit- 
tees to play a more prominent role in implementing the 2010 Cen- 
sus. 

Senator Carper. OK. Talk about that last one again just a little 
bit more, please. 

Mr. Hill. These Race and Ethnic Advisory Committees are really 
very important, because they have members who come from diverse 
minority communities and work very well together. For example, 
we supported the recommendations of American Indians, and His- 
panics and vice versa. They are also strong advocates for their local 
communities. 

One of the most effective ways of reducing the Census 
undercount is to have messages that are communicated by people 
who are trusted at the local level. That is the main ingredient — 
that local people are used who come in contact with others at their 
level, not at a higher level, and who can communicate the message 
to them. We have found this strategy to work every single time. 
These representatives can be very effective in many ways, such as 
distributing foreign language Census forms and helping people to 
complete their forms at assistance centers. 

All of these activities are important for reducing the non-re- 
sponse follow-up. High response rates to mailed questionnaires will 
reduce the extent to which non-response follow-ups — which are the 
most tedious part of Decennial Censuses — are needed. 

Senator Carper. I think one of you, I don’t know if it was Mr. 
Thompson, but one of you testified that the non-response, I think 
we had more people responding, was it in 2000 than in 1990, by 
2 percent? It actually went up, the number of respondents went up, 
I think you said it was 65 to 67 percent. Good. 

Others on the panel, if anybody would like to say that you think 
Dr. Hill has some good ideas, if you do, that is fine. Say that. If 
you think there are some other ideas on it, some of you made rec- 
ommendations that were similar, others different, but I would love 
to hear your ideas. Let us start with you, Ms. Bryant. 
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Ms. Bryant. I agree completely on the paid advertising. 

Senator Carper. I think you mentioned that. 

Ms. Bryant. We did not have it in 1990. The Bureau went one 
Census too long, and I am sure Mr. Thompson will agree with me, 
on depending on Public Service Announcements. The TV and radio 
stations were no longer required for their FCC licenses to give you 
around-the-clock, so we had some wonderful advertisements, in- 
cluding using four minority advertising firms, but they were play- 
ing at 3 o’clock in the morning. And so going to the paid adver- 
tising, which was a recommendation after our Census, I think 
made an enormous difference. And, of course, I am very attuned to 
Census things, but I just heard them everywhere. It was a really 
big campaign and I know it cost a lot of money, but you have got 
to pay for that. 

Senator Carper. Some of you remember the battle between the 
States and the tobacco industry a decade or so ago when the States 
attorneys general sought to extract a fair amount of money from 
the tobacco industry and a lot of that actually went into a founda- 
tion called the American Legacy Foundation, whose job it was to 
try to transmit to young people in this country the message not to 
smoke, not to get started, and if you are, stop. 

I was the founding vice chairman of that as governor at the time 
and we decided we would do a paid ad campaign. We decided it 
wouldn’t be ads that guys like me would develop, but we would find 
really younger, hip ad agencies who could connect with young peo- 
ple, and they ended up putting their messages on TV shows that 
I never watched, but my sons later did. A lot of young people did, 
music stations, radio stations and so forth, the Internet, in ways 
they are just a lot smarter to connect that. It sounds like that is 
what we need to do here today. 

Ms. Bryant. Well, similarly with the African American other 
group agencies, they know the media their people watch. They did 
a great job in our Census of producing ads that just — they weren’t 
given enough air time because we didn’t pay for it. 

Senator Carper. Good. All right. Mr. Thompson. 

Mr. Thompson. Well, I agree with Dr. Hill. I would, as a foot- 
note, note that the reason that we had such an active partnership 
program in addition to the paid advertising in 2000 was the edu- 
cation that the Racial and Ethnic Advisory Committees provided to 
us on the importance of reaching out at the grassroots level to in- 
still participation. 

I would, however, add a little bit to his recommendations. I think 
the most important recommendation I made, and I will restate it, 
is that there needs to be a thorough, extensive field test of all the 
systems involved in their non-response follow-up operation. That 
operation is the key to a good Census. 

Senator Carper. OK. Good. I think a couple of other witnesses 
have said the same thing. 

Maybe one more, Mr. Goldenkoff, and then I will yield to Dr. 
Coburn. 

Mr. Goldenkoef. Sure. I think that everything that has been 
said thus far is consistent with what GAO has said in the past, but 
I would like to put a couple of other things out on the table. 

Senator Carper. Please. 
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Mr. Goldenkoff. Leadership. In addition to the timely appoint- 
ment of a Census Director, what should that Census Director do 
when he or she gets in office? One, I think it is fairly important 
that the Census Director regularly reach out to key House and 
Senate committees. Transparency is key. It helps ensure that the 
Census is on track. It builds that confidence that we talked about 
earlier. It is so important, especially when you have an operation 
that is going to cost between $14 and $15 billion. The Director also 
needs to embrace oversight. Transparency is very critical because 
that also instills a comfort level. 

Operationally, risk and cost management are essential. We need 
to ensure that the IT systems are fully tested. That, we have spo- 
ken at length about. The Bureau also needs to set priorities. 

And I would also like to put a slightly different take on some of 
the outreach and promotion activities as well. We agree with the 
partnership specialists and need for partnerships in general. A 
Census is inherently local. But I also think the Bureau should con- 
sider other ways of using partners for other types of operations. 
For example, in the 2000 Census, the partner — I forget which local- 
ity it was, but the locality actually enlisted the help of their trash 
collectors because they cover the streets every single day and the 
locality trained them to look for hidden housing units. So that was 
a case where they used another local group, part of the government 
was enlisted to help in an address-building operation. 

Building a blog. Maybe the Census Director could consider put- 
ting a blog up there to deal with snafus that might come up or pro- 
vide regular progress on the Census on a daily basis. But again, 
part of just a different take on the outreach. 

Senator Carper. That is a great list. Let me yield to Dr. Coburn. 
Thank you all for your responses. 

Senator Coburn. Well, thank you for your testimony. Mr. 
Thompson, I am not wanting to put you on the spot so I am going 
to ask this question where you don’t have to answer it directly and 
then I will talk to you later. [Laughter.] 

One of the eight things you said you all did in 2000, number 
seven was that you had a strong management team in place. Are 
you in a position now where you could assess whether or not there 
is a strong management team? I am not asking whether there is 
or not. I am just saying, are you in the position now where you 
could make that assessment of what you see at the Census Bureau 
now, since you are working with them so closely? 

Mr. Thompson. I believe that I could provide an assessment. 

Senator Coburn. OK. I will let you off with that. I don’t want 
to put you on the spot. 

Mr. Goldenkoff, you talked about risk management systems that 
need to be in place, and I have not seen your report yet, so I am 
operating at a deficit if it is out there. My staff has seen it and I 
didn’t get a chance to thoroughly prep for this hearing. Have you 
all specifically listed those areas where they do not have now and 
need to have 

Mr. Goldenkoff. I will turn it over to Mr. Powner. We did make 
10 recommendations in the report that we issued today. 

Senator Coburn. Yes, but it is just 10 and it is in the report. 
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Mr. PowNER. Yes. It is very detailed, Dr. Coburn. There are six 
systems that need to be tested 

Senator Coburn. I have got that down 

Mr. PoWNER [continuing]. There are 44 operations 

Senator Coburn [continuing]. And the integration 

Mr. PoWNER [continuing]. And there are about 250 interfaces 
that need to be prioritized. At one time, they said they were going 
to designate a test director. We have someone who is working in 
that position part-time. We address that in the report. We have a 
recommendation for a dedicated test director. And then clearly we 
need metrics in place to really monitor this going forward because 
there is a lot to get done here. 

Senator Coburn. OK. And in your report, you are recommending 
that they have to list the areas of interfaces and then test them? 

Mr. PowNER. Yes, absolutely. 

Senator Coburn. You make that absolute recommendation. They 
see it, and they know it is there. 

Mr. PowNER. Absolutely. 

Senator Coburn. So in your opinion, if they follow your rec- 
ommendations, both in terms of management, technical achieve- 
ments, risk intervention, preparation for risk failure, testing, 
should they be able to accomplish what they need to accomplish for 
the 2010 Census? 

Mr. PoWNER. From a systems point of view, I think it is still 
highly likely they won’t be able to test everything completely. 

Senator Coburn. Before they go into the field? 

Mr. PoWNER. Correct. 

Senator Coburn. How about while they are going into the field? 
In other words 

Mr. PoWNER. You could continue, sure. You can continue while 
you are live, and frankly, that is what happens. I mean, when 
something goes wrong, you have bugs and you fix them on the fly, 
right? 

Senator Coburn. Right. 

Mr. PoWNER. But clearly, that is why the need here is to really 
prioritize. I mean, there is systems, the integration and the oper- 
ations. One of the most important operations we heard, NRFU with 
the operational control system, that all needs to be tested collec- 
tively. That is clearly one of the key operations. But there are also 
others. So prioritization and really having these plans in place. 

But I think it is likely they won’t get to some of them. That is 
why we want to see that prioritization. 

Senator Coburn. I had a conversation with a CEO of a firm, not 
this particular firm, that makes one of these. He said, in 3 months, 
they could have put together a package that you could use for 
NRFU to do everything they want and transmit. They never were 
asked. Never were asked, not once. They didn’t ever go to anything 
outside the contract they had. Even once they got in trouble, they 
never went to look, is there a way where we can still solve this, 
save money, have collection of data, transmit it. It was never 
asked. 

Even if we get a new Census Director, we have 20th Century 
thinking, in my estimation, at the Census Bureau, not 21st. The 
resistance to online, as Dr. Brown talked about, I mean, I have 
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been fighting this for 4 years, ever since they abandoned the Lock- 
heed contract. Sorry, we are just not going to do it. 

And so I have great worries. The number one worry, this isn’t 
going to cost $14.8 billion. It is going to cost $18 to $19 billion. You 
wait and see. There will be another $3 or $4 billion in an omnibus 
bill, emergency bill for the Census because, oh my gosh, we can’t 
get it done. And part of the bureaucracy is don’t ask for everything 
you need now because if you ask it under emergency for a Census, 
you are backing up against the window and you are going to get 
it. So it is going to cost — is it $9.3 billion, is what the 2000 Census 
cost in today’s dollars, and we are going to be at least double that. 

Mr. Hill, tell me how we do — I understand the organization of 
the African American community. It is very well organized in a lot 
of these. How do I do it for Native Americans in Oklahoma? Tribal, 
yes, where we have reservations, it is much easier because we have 
an isolated group. But in States like Oklahoma and Tennessee and 
some of these other States that have large tribal populations but 
they are not reservation-based, do you have any ideas on how that 
outreach can be best accomplished? 

Mr. Hill. You are right about the greater difficulty of reaching 
non-reservation American Indians. There needs to be more aggres- 
sive outreach and targeting of community groups who work closely 
with Native Americans who do not live on reservations. The cur- 
rent approach appears to be fragmented and not reaching the 
grassroots groups. There is an urgent need to more effectively tar- 
get community groups and tribes who work closely with American 
Indians not living on reservations. This strategy can work if strong 
local partnerships are developed with such indigenous groups. 

When I talk about a more prominent role for members of the 
Race and Ethnic Advisory Committees, there are many things that 
can be done by working from the bottom up. This community-based 
approach is especially needed for minorities who speak foreign lan- 
guages. 

Senator Coburn. In other words, it is important to ask the ques- 
tion, will you come help us? 

Mr. Hill. That is right. 

Senator Coburn. Now tell us how. 

Mr. Hill. As I said before, strong partnerships can markedly im- 
prove outreach to the various minority groups, even among Amer- 
ican Indians who do not live on reservations. 

Senator Coburn. OK. I am almost out of time. Mr. Brown, there 
was imputation used in the last Census. Would you comment on 
that, whether or not it resulted in overcount or undercount for 
where it was utilized, in your estimation, and whether or not that 
is an appropriate thing to do in this Census. 

Mr. Brown. I think imputation is essential in any Census. There 
are many situations in the field where the enumerators cannot talk 
directly to the resident of the household, and yet they know it is 
occupied. So one way or another, the data has to be included, from 
neighbors — that is part of my emphasis on administrative records, 
is that those provide potentially a much better method for filling 
in data in households you know are occupied. 

Senator Coburn. Do you know whether or not that would comply 
with the Supreme Court ruling? 
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Mr. Brown. No, and I am not a legal expert 

Senator Coburn. OK. Does anybody on the panel know whether 
that would comply with the Supreme Court ruling? 

Mr. Goldenkoff. I believe it does. I don’t think there was a 
court decision that disputed it. 

Mr. Thompson. I believe there was — Utah raised a lawsuit about 
the use of imputation. It did go to the Supreme Court, and I believe 
that it was found to be acceptable. 

Senator Coburn. OK. Thank you. Please continue. I am sorry to 
interrupt you. 

Mr. Brown. So I do think imputation is essential. It needs to be 
done carefully. As a statistician, I think that statisticians could 
suggest a lot of improvements in the current imputation method- 
ology. Much of the duplication that Mr. Carper mentioned is re- 
lated to imputation processes, and so there is plenty of room to im- 
prove on this product, but I think it is needed in some form or 
other. 

Senator Coburn. Thank you. 

Mr. Chairman, you have been very generous with time. Thank 
you for holding this hearing. I would make one comment before I 
have to leave. I think it is real important that your Subcommittee, 
even if we don’t 

Senator Carper. I would say, our Subcommittee. We used to call 
it our Subcommittee. 

Senator Coburn [continuing]. Our Subcommittee, even if we 
don’t have a Census Director named, whoever is acting and who- 
ever is along before your committee on the basis of the rec- 
ommendations of the GAO and also what we heard here today and 
see where the planning is ongoing. 

Senator Carper. Good. I think that is a very good recommenda- 
tion. 

All right. Senator Burris. 

Senator Burris. Thank you, Mr. Chairman. 

Senator Coburn, your last statement was where is the Acting Di- 
rector, would be my question, and we should certainly have that 
Acting Director here, Mr. Chairman. 

Second, Mr. Chairman, I would also give you a name of a person 
who can move in that direction in terms of running this operation 
because right now, evidently from what I hear from GAO, they are 
really behind the eight-ball in terms of what direction will come in 
order to get this off the ground in a timely fashion. 

But I am concerned with Dr. Hill’s testimony in terms of partici- 
pation of minority groups, and since the Acting Director and you 
all are not really from the Census Bureau, it wouldn’t do any good 
to say that what Dr. Hill is saying is what is needed to make sure 
that we don’t get an undercount. We need them involved, every 
group involved in every level of the Census steps, from the plan- 
ning and the implementation, so that we can make sure that the 
Indians are counted, the Asians are counted, the Hispanics are 
counted, even the Appalachians are counted. We need to have ev- 
eryone counted and my assessment of that is that there is a way 
of doing it and we ought to make sure that we also use minority 
contractors in order to reach those individuals. When all those 
monies are going to be spent, some of those dollars are going to 
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have to be given to the advertisers and marketing people and their 
various groups. 

So any comments on that? Ms. Bryant, you ran that. 

Ms. Bryant. Well, I think our Census was the first one — Vince 
Barabba is not here to argue with me — that did use minority ad- 
vertising contractors to produce the spots. As I say, we did not 
have paid advertising, but they produced very excellent spots that 
were 

Senator Burris. They just didn’t have time to run during the 
regular 

Ms. Bryant. Well, they ran, but they ran at 3 o’clock in the 
morning. 

Senator Burris. Yes, I heard you say that. 

Pardon me. Dr. Hill, do you want to comment on that? 

Mr. Hill. Yes. In the 2000 Census, we used paid advertising for 
the first time and it was very effective because the REAC Commit- 
tees recommended a number of these firms. Each of the minority 
firms targeted specially-prepared messages to their respective 
groups. For example, the African American ad firm not only tar- 
geted messages to African Americans, but also to black immigrants 
from the Caribbean and Continental Africa, such as Haitians, Nige- 
rians, Jamaicans, Trinidadians, etc. Thus, numerous focus groups 
were held with black non-immigrants and immigrants to develop 
culturally-sensitive messages. 

Senator Burris. Sure. 

Mr. Hill. The 2000 Census demonstrated that paid advertising 
works, since minority subcontractors targeted specially-developed 
messages for their groups. Paid advertising can be even more effec- 
tive in the 2010 Census. 

Senator Burris. Are you still involved with the Census 

Mr. Hill. No, I am not currently a member of the 
AfricanAmerican Advisory Committee. I cycled off a few years ago. 

Senator Burris. Who is replacing you in terms — is there a Dr. 
Hill in the Census Bureau? [Laughter.] 

Mr. Hill. I am not familiar with the current members of the Af- 
rican American Advisory Committee. But those members usually 
span a cross-section of persons who represent many sectors, such 
as ministers, community-based groups, grassroots leaders, etc. 

Senator Burris. OK. Then, Mr. Powers, while technology has led 
to ambitious steps to administer the Census, are there any new 
ways we can employ to combat undercounting typically underrep- 
resented — any technology. Is there any technological emphasis that 
we could use? 

Mr. POWNER. Well, I think clearly if you could have used the 
hand-helds for the non-response follow-up, I mean, any use of tech- 
nology for going after the folks via non-response follow-up, there is 
potential there. We are back to paper-based operations there. So 
that is one area that you could potentially use 

Senator Burris. Well, Senator Coburn just raised his cell phone 
and said that technology could be used. Is there some kind of way 
we can get that information to the Acting Director so that they can 
start looking at what technology would be there available for 
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Mr. PowNER. What happened there, there was a plan to use that 
and then that contract ran into many problems that Chairman 
Carper looked at in great detail. 

Senator Burris. Oh, OK. 

Mr. PowNER. And what we did is reverted hack to things they 
knew how to do with paper. But going forward, I think that is a 
very valid point, is we need to look at using technology 

Senator Burris. So can we look at that now? We have about 18 
months, you said, isn’t it? No, 12 months 

Mr. PoWNER. I think now from a risk mitigation point of view, 
you probably want to stay the course, but you want to look at that 
for 2020. We mentioned the Internet. We mentioned using hand- 
helds for more operations. I mean, we need to start thinking ahead 
for that. But right now, we are kind of in emergency management 
mode and I am not certain we would want to introduce that. 

Senator Burris. I mentioned the fact, too, that people might be 
afraid of their identity being stolen. Is there any technology that 
would try to ensure that when they give this information, it would 
say that it won’t be given to the Social Security Administration or 
the IRS? Individuals are going to be a little skeptical now that 
their identity could be swiped so easily. So is there any technology 
that we are looking at to protect that? 

Mr. PoWNER. Well, clearly, I think there are human processes 
you want to have in place to protect identity theft. But from an in- 
formation security point of view, all these systems need to be ac- 
credited and certified as secure to help avoid that situation. 

Senator Burris. Thank you very much. Thank you, Mr. Chair- 
man. I appreciate it. 

Senator Carper. Thank you. Senator Burris. 

At least one of our witnesses mentioned the importance of having 
a set term, maybe 5 years, for a Census Director, and I could think 
of arguments for doing that and I could think of arguments not to 
do that, but I would appreciate each of our witnesses saying wheth- 
er or not you think that is a good idea. I think our Commissioner 
for the IRS, as I recall, now serves a 5-year term. We have some 
other positions where people serve terms, not uncommonly a 5-year 
term. The idea is to overlap from one Presidential term into the 
other to carry over. What do you all like about that idea, or on bal- 
ance, what do you think you don’t like? 

Ms. Bryant. Well, since I am the one that brought it up 

Senator Carper. I thought you did. 

Ms. Bryant [continuing]. The problem is the 10-year cycle of the 
Decennial Census and the 5-year cycle of the Economic Censuses 
is just out of tune with a 4-year cycle for the current appointment. 
As I say, I am the worst example since I got into office 3 weeks 
before the Census. 

Senator Carper. I hope you were a quick study. I bet you were. 
You needed to be. 

Ms. Bryant. I sure worked hard at it, with the help of John 
Thompson, I might say. He was my tutor on some of the statistical 
parts. 

Senator Carper. All right. Thank you. 

Mr. Thompson, having tutored Ms. Bryant at an earlier stage in 
your life, what do you think of her idea? 
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Mr. Thompson. She is being very kind about who tutored who. 
I think that it is a very good idea to have a term appointment for 
the Director of the Census Bureau. It is very unsettling when you 
have one Director leave and another Director come in. There is a 
period where you don’t have leadership, and understanding exactly 
when that will happen and having a term, I think is very good. 

Senator Carper. Any downside? 

Mr. Thompson. I don’t see any downside. 

Senator Carper. All right. 

Ms. Bryant. I will interject that this is being supported now by 
all seven past living Directors. 

Senator Carper. All right. 

Ms. Bryant. So bitter experience has told us this would be a 
good idea. 

Senator Carper. Thank you. 

Mr. Goldenkoff, do you have any thoughts? 

Mr. Goldenkoff. It has the potential to provide that continuity 
that is so important. As we have said, the Census cycle, it is at 
least a 10-year cycle, so you need someone who is not really going 
to be a temporary employee, someone who is in and out. And I am 
not even sure of all the past Census Directors how many of them 
served as long as 4 years. If so, it is relatively rare. And so you 
need someone, certainly someone in place who has that continuity 
and has longer-term vision. 

Senator Carper. Thank you. Any downsides you can think of? 

Mr. Goldenkoff. Well, to the extent that — I am thinking about 
the legislation that has been proposed to make the Census Bureau 
an independent agency, of which that is a provision. It doesn’t nec- 
essarily follow that having a 5-year term of office would make the 
Census Bureau independent. I mean, there are other factors in 
play, and even those agencies that have a fixed term of office, they 
are just as susceptible to political influence, partisan influence, as 
agencies with a tenure that follows the President’s. So it is also a 
function of the personality of the Director, so — it is not a panacea, 
is my point. 

Senator Carper. In the Department of Treasury, we have the 
Commissioner of the IRS, so you have a good point. 

Ms. Bryant. And Director of the Bureau of Labor Statistics, the 
other big statistical agency. 

Senator Carper. There you go. But I think maybe in each of 
those instances, certainly in IRS, the Department of Treasury, you 
have a Commissioner serving a 5-year term and the head of the 
Department is a political appointee who may not be there for 5 
years. 

Mr. Powner, any thoughts on this? 

Mr. Powner. Yes. I think the continuity of the leadership is key 
and would be very supportive of that. We do a lot of work for you, 
Mr. Chairman, looking at the management of the IT budget, $70 
billion spent across all Federal agencies. This has been looked at 
with Federal CIOs. So if you are a political appointee, CIO, your 
average tenure is less than 2 years. If you are career, it is slightly 
over 2 years, but still less than 3 years. And I think it has been 
well accepted in that that 4 to 5 years would be much better to 
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have the continuity from a technology point of view and I think it 
applies here, also. 

Senator Carper. Good. Thank you. Dr. Brown. 

Mr. Brown. So let me remark on that from my perspective. Con- 
tinuity of leadership is really very important and I think you men- 
tioned, or maybe Mr. Coburn, the fact that in many respects, the 
Census Bureau is a 20th Century vehicle operating in the 21st 
Century. I think part of the problem has to do with the lack of con- 
tinuity in leadership, both at the top of the Bureau and a little bit 
further down in terms of research and development. 

Senator Carper. Thank you. Dr. Hill, any thoughts on this idea 
of a 5-year term for the Director? 

Mr. Hill. I support it. Most critical decisions for the Census 
occur between 3 to 5 years before that Census. I think a 5-year 
term is in a good direction, and anything that would make it as 
nonpartisan as possible is preferred. 

Senator Carper. OK. How about the idea that the Director of the 
Census should report directly to the President? I think that is in 
some legislation that the House is considering. And we have had 
it in — this Subcommittee is part of the Committee on Homeland 
Security and Governmental Affairs. Part of our jurisdiction is 
FEMA, and there has been a lot of discussion since Hurricane 
Katrina that the head of FEMA should report directly to, not to the 
Secretary of Homeland Security, but should report directly to the 
President, and that is a debate that is probably still going on in 
some circles. So it is not an argument that we hear just in the in- 
stance of the Census Bureau. 

But do we need to have, in your judgment, a situation where the 
head of the Census reports to the President as opposed to the Sec- 
retary of Commerce? 

Ms. Bryant. Well, having the Census Bureau as an independent 
agency, as the National Science Foundation is, for example, an- 
other apolitical type of organization, would remove two layers of 
bureaucracy that the Census Director or anybody at the Census 
Bureau has to go through in order to talk to you in Congress, to 
talk to the press, and I am one for flattening organizations. 

Senator Carper. Mr. Thompson, do you have any thoughts? 

Mr. Thompson. I really don’t have an opinion on that. 

Senator Carper. Fair enough. Mr. Goldenkoff, any thoughts? 

Mr. Goldenkoef. I think there would need to be some safe- 
guards in place, with the Census, it is important to have impartial 
data, and so anything, even the appearance that there was some 
type of political influence going on could really undermine the 
credibility of Census data. So I think that you would want some 
type of White House oversight or a connection there. It should be 
on the management and operations, but it should stop at anything 
that has to do with the science of taking the Census. So it is just 
finding that right balance and having the appropriate safeguards 
in place and I think that is really what is critical. 

Senator Carper. All right. Thank you. Mr. Powner, any 
thoughts? 

Mr. Powner. I think you could be effective with either scenario. 
I think what is most important, though, is to get the right leader- 
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ship and the right management processes in place, and that has 
heen the primary issue with the Census Bureau. 

Senator Carper. Thank you. Dr. Brown. 

Mr. Brown. I have really no thoughts about this. 

Senator Carper. OK. Dr. Hill, any thoughts? 

Mr. Hill. I agree with Mr. Goldenkoff that more balance is need- 
ed. I am not certain whether it should be separate or part of the 
Commerce Department. I just think that, however it is structured, 
it should have the freedom to make independent programmatic de- 
cisions. 

Senator Carper. All right. We have operated in the time I have 
been here in preparing for the upcoming Census where a Secretary 
of Commerce oversees his empire, which includes the Census Bu- 
reau, and there was a time when we realized, sitting in this room, 
the Secretary of Commerce wasn’t very mindful of those respon- 
sibilities, his oversight responsibilities in terms of providing direc- 
tional leadership for the Bureau of the Census. 

Once we got his attention and once he focused on that responsi- 
bility, he was a great asset, going to the Administration, 0MB, the 
President, and saying, we need extra resources, and some of you, 
I think Dr. Hill and others, said one of the primary recommenda- 
tions you have is make sure we have the right resources to go out 
and do the job well. Once Secretary Gutierrez, to his credit, real- 
ized we had a problem here, he got engaged and helped us move 
that and worked with the appropriators to make sure we had the 
resources that were believed to be necessary. So it actually can be 
helpful in that regard. 

Ms. Bryant. Well, when I talk about we are removing two levels 
of bureaucracy, that also means between the Census Bureau and 
the 0MB, so another place. 

Senator Carper. All right. Thank you. 

Others have sort of talked around this question and I just want 
to come back to it one more time. I think in Senator Burris’s ques- 
tioning, he was asking Mr. Powner or Mr. Goldenkoff about actu- 
ally acting now to better ensure that the technology that Dr. 
Coburn believes is available, that we actually put it to use now to 
get a better outcome for 2010. I think your response was, well, 
right now, let us just do what we said we were going to do and do 
that well, implement that well, test out the systems that still need 
to be tested out rather than starting anew. 

Any advice for us as we go through the next year or two doing 
well, counting well, accurately, and in a cost-effective way the peo- 
ple who live in this country, but is there anything that we ought 
to be mindful of doing to better ensure that when we get to 2020, 
we are not doing a Census that is at least part pencil and paper 
again? 

Mr. Goldenkofe. I would agree with some of the comments that 
were made earlier. If you look at the way the Census is being con- 
ducted today, with the exception of some improvements in tech- 
nology, it is basically the same approach that has been used since 
1970. We talked about there were some changes in advertising, but 
essentially it is a mail-out, mail-back operation and that approach 
has really exhausted its potential to count the Nation cost effec- 
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tively. The Bureau has to spend more money, work harder to get 
essentially the same result. 

So I think that we really need to look at what other approaches 
are out there that can either get better results or at least control 
the costs better, and whether that is administrative records or new 
uses of technology, maybe some of the rules of the Census need to 
be reexamined given changes in society. Does it make sense to 
knock on a door six times during non-response follow-up? You have 
reached the point of diminishing returns on that. All these things 
probably need to be on the table and should be reexamined. 

Senator Carper. All right. Any other thoughts on this? 

Ms. Bryant. Well, I think a lot of people would respond on the 
Internet and we wouldn’t have to do all this mail-out, mail-back, 
so forth and so on. However, I am not sure that will help on reduc- 
ing undercount because there is a skew on who uses the Internet. 

Mr. Brown. That is true. There is at least one aspect that has 
been mentioned to us where it could be quite helpful because if you 
have Internet response option, it is easy to incorporate Hispanic, 
Spanish and other foreign languages 

Ms. Bryant. Languages, yes. 

Mr. Brown [continuing]. And it also can be easier for proxies to 
help people fill out and respond. So there are ways in which the 
Internet can help, although the primary target population is prob- 
ably — it is probably more an issue of cost saving than response im- 
provement. 

Senator Carper. OK. All right. Thank you. 

Given the cost of the Census in an era of unprecedented fiscal 
challenges, what are the cost drivers of the Census and how can 
the Bureau produce an accurate yet cost-effective Census? Anyone 
at all? 

Ms. Bryant. Well, unfortunately, cost cutting was precluded by 
that Supreme Court decision because the most expensive thing is 
going after the non-respondents. The design of 2000 included sam- 
pling the non-respondents and estimating the rest and that got 
shot down by the Congress sending up a bill to the Supreme Court 
that the Supreme Court supported. 

Senator Carper. Well, it sounds to me like part of what you all 
have said, a number of you said, one, make sure that — the key here 
is to try to reduce the number of people that aren’t responding. A 
good ad campaign properly conceived and implemented, I think can 
help on that. The idea of these partnerships that we talked about 
earlier and making sure that if we are interested especially and we 
have low response rates from, we will say, African Americans, from 
Latino Americans, from Native Americans, to make sure that we 
are involving, I think as Dr. Hill said, make sure that we are in- 
volving folks maybe who do public relations campaigns, public out- 
reach campaigns, to make sure that we have included in folks who 
are formulating those campaigns folks whose background is maybe 
similar to those ethnic groups. 

Mr. Goldenkoff. I think one — I would like to add to that a little 
bit, though. We know what some of the major cost drivers are, non- 
response follow-up probably being the largest cost driver. One of 
the things that the Bureau has — can do a better job of is identi- 
fying where it gets the most bang for the buck. It has a number 
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of repetitive operations, and it is probably a good thing to have 
some redundancy in addressing building operations. However, in 
2000, I think there were about a dozen different operations to build 
the address list. Well, do you need all of them? 

I think what the Bureau has not really done a good job of is iden- 
tifying where it gets the most results from, and that is true with 
the way it builds the address list, advertising, where does it get the 
most bang for the buck in terms of advertising. Is it paid adver- 
tising or is it through very locally-targeted partnership efforts? So 
maybe that is something the Bureau should be thinking about now, 
is how to evaluate, working on evaluation so that come 2020, they 
will have a better idea of where to invest their resources. 

Senator Carper. That is a very good point. 

Any other thoughts on this question? All right. 

I have two more, two more to go. In April, like next month, the 
Bureau is scheduled to begin its address canvassing. That is an op- 
eration that in total requires some 140,000 temporary workers who 
rely on hand-held computers to verify addresses and map informa- 
tion to update the Bureau’s master address file and digital maps. 
And this maybe should be as much a question for our friends from 
GAO as not, but are you confident that the hand-helds will perform 
as expected? 

Mr. Goldenkoff. I think that we have more confidence than we 
did last summer. The Bureau conducted an operational field test 
back in December. It was in Fayetteville, North Carolina. And cer- 
tainly some of the problems and issues with the data transmission, 
the unreliability, did not recur in Fayetteville. But what the Bu- 
reau demonstrated by that in a large degree was that under condi- 
tions similar to Fayetteville, the hand-helds will work. 

Obviously, the country is very different in respects from Fayette- 
ville. There are urban areas. There are suburban areas. And that 
is the big unknown right now. And so they have made progress, 
and I think that is commendable from where they were back in the 
summer, but overall success is still an open question. 

Senator Carper. All right. Anybody else on this one? 

All right. The Administration’s fiscal year 2010 budget doesn’t 
propose increasing recruitment and hiring of Census takers for 
non-response follow-up as Congress envisioned in approving extra 
funds for the Census in the stimulus package. One of you men- 
tioned, I think, $1 billion was added in the stimulus package for 
this purpose. Would the Census Bureau be better able to meet the 
challenge of lower-than-projected mail response if it could recruit 
and hire additional field staff heading into peak Census operations 
in 2010? 

Mr. Hill. Yes. We keep coming back to the non-response follow- 
up. One of the biggest obstacles to an accurate Census count is the 
high turnover of enumerators. However, the depressed economy 
should result in an influx of Census workers and greater stability 
and continuity. A major priority of the 2010 Census should be to 
increase the continuity of Census takers and to reduce the historic 
high turnover rates, especially in inner-city areas. 

Senator Carper. Yes, sir? 

Mr. Thompson. I think this would be a really good question to 
ask the Census Bureau for the following reasons. They do have 
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plans in terms of they understand how many people they have to 
recruit, how many they have to hire. They understand pay rates. 
But it would be good if they would come before you and explain 
their assumptions and rationale so that you would feel comfortable 
that they have thought through the process or the numbers that 
they have and what they have budgeted for the recruiting and hir- 
ing. In 2000, we had enough funding from the Congress that we 
were able to put that operation together. 

Senator Carper. All right. One last thing. Do you all have any- 
thing you would like to add? We had, I think, a very good discus- 
sion here. Anything that comes to mind that you say, oh, I wish 
I had said this? Usually, when I walk out of here, I think, boy, I 
wish I had said that or asked something else. Anything you all 
want to add as take-away? No? OK. 

A couple of thoughts. Let me just kind of wrap up what I think 
I have heard here. One is everybody says we need a first-rate Di- 
rector of the Census Bureau and we need him or her right now. 

Second, I think what I have heard is, for the most part, people 
say we want to make sure that whoever is leading this operation 
has the resources, human resources and financial resources and 
technology resources to do the job as best we can, to make sure we 
count as closely as we can the number of people who live in this 
country and do it in a cost-effective way. 

I think I have heard here that we maybe ought to give serious 
consideration to whoever is going to be serving as our Census Bu- 
reau Directors in the future to be nominated and confirmed to 
serve a multi-year period of time, maybe 5 years, maybe something 
more or less. Sort of a mixed bag in terms of whether or not we 
need a direct report from the Census Bureau directly to the White 
House. 

I take away from here the importance of having an ad campaign 
and a well thought-out ad campaign, particularly involving in the 
creation of the ad campaign folks who can better design the cam- 
paign to go after our target audience of people that aren’t respond- 
ing. 

I mentioned earlier the American Legacy Foundation where I 
was privileged to serve as their founding Vice Chairman back in 
the late 1990s, right at the turn of the century, and the folks who 
developed the advertisements to young people were not, as I said, 
adults, not for the most part. Actually, the ideas came from the 
kids. Literally, the ideas came from the teenagers and they worked 
with ad agencies, but the raw product ideas came from the kids. 
The testing was on the kids, teenagers and so forth, even younger 
than that. But that is a message or a lesson that I am taking away 
from here. 

A couple of others, as well. I won’t go any further, but I think 
just a lot of good reminders here. I was talking to a friend of mine 
today about basketball and he was talking about more games are 
won in the planning of the game. I think he was talking about 
Bobby Knight. Remember Bobby Knight at the Indiana University, 
later was at Texas Tech — was it Texas Tech? Was that where he 
went? But he didn’t always have the best team on the floor, but 
he always had the best plan going into a game of just about any- 
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body. That is why he was so successful. I think having a good plan 
here is valuable. 

I think another good idea that we had was that a good deal of 
our operation — the program hasn’t been actually tested operation- 
ally and we have a fair amount of work still to do there before we 
are ready for prime time. A whole bunch of good ideas. 

One of the things I am inclined to do, I have asked you to pro- 
vide for me two good names by close of business tomorrow of folks 
that we could submit to the Administration as a talent pool in case 
they need some help in that regard. 

The other thing I might do, once we have got somebody who has 
been nominated, vetted, confirmed, in office, we might want to pull 
you back together again, maybe just on the phone, not even in per- 
son, maybe just do it on the phone in some kind of teleconference 
call to spend some time with some of us, our staff, the new Direc- 
tor, maybe a person or two from his or her team, just to go through 
again some of these points. I just think that might be time well 
spent. And if you might find time to do that with us, I would be 
grateful. 

And you are going to get a couple of questions from people who 
are Members, some who were here and some who weren’t, but 
some follow-up questions. We would ask that you respond to those 
promptly. I think we are going to leave the record open for 2 weeks 
for that, so if you get any follow-up questions from us, please re- 
spond to them promptly. 

Are we forgetting anything here? All right. It has been a very 
good hearing, timely, and I think most informative, and we are 
grateful to you for spending your time with us and thank you for 
your ideas and input and for your willingness to give us some help 
between sundown now and sundown tomorrow, and also maybe 
your willingness to join us in a conversation with the new Director. 
Hopefully, he is going to be identified, vetted, confirmed soon. 
Thank you very much. 

With that, this hearing is adjourned. 

[Whereupon, at 4:11 p.m., the Subcommittee was adjourned.] 
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Mr. Chairman and Members of this Senate Subcommittee, thank you for inviting 
me to participate in this hearing. The most important lesson i learned while 
serving as Census Bureau Director, is that it takes a lot more than the resources 
of the Census Bureau to conduct the constitutionally mandated “Census of 
Inhabitants of the States.” 

Title 1 3 of the U.S. Code makes it clear that the Census Bureau has the primary 
responsibility for the planning and the conduct of the census. But in our free and 
diverse society there are very few endeavors that require the participation of 
every household. As a result the scale of this undertaking contributes to the 
extreme complexity and difficulty of counting our population. 

One of the ways we can appreciate and understand the complexities and 
difficulties of taking the Census in a free and diverse society is to compare it to 
census taking in places where individual freedoms are sometimes put aside for 
the purposes of efficiency and accuracy. Such a comparison was made in an 
article describing the taking of the Turkish Census that appeared October 23, 
2000 in the Washington Post; 

“Virtually all of T urkey was under a dawn-to-dusk curfew today as 950,000 
government workers fanned out cross the country to take the official 2000 
door-to-door head count. No one was allowed to leave home from 
5:00a.m. Until about 7:00p.m., so the streets of the usually jampacked city 
of 10 million people were eerily deserted. 

Threatened with three-month jail sentences if caught off their property 
without permission, fishermen hung up their nets, traders at Istanbul’s 
famed bazaars shuttered their stores and barely a car could be found 
moving on the usually traffic-choked streets.” 

Under the imposition of such harsh rules, which are unimaginable in this country, 

I believe the cost of a census would decrease and it is likely the accuracy of the 
count would improve. But whatever the savings in Census Bureau budget items, 
the increased costs to society in both economic terms and loss of personal 
freedoms would, of course, overwhelm any of the benefits. 

Maintaining the freedoms we cherish and the need to let the country operate at 
full speed during the taking of the Census requires the cooperation of the 
“inhabitants of the States” and their elected leaders - as well as leaders from 
other areas of our society. 

I would suggest that there is a relatively simple way for elected and appointed 
leaders of the Executive branch and the elected officials of the Senate and the 
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House of Representatives to make a major contribution to the taking of the 
Census. It is as simple as devoting a considerable portion of attention on 
encouraging the citizens of our country, through direct and personal contact, to 
fill-out and send back the census form as quickly and accurately as possible. 

I had the opportunity to serve as Census Director during the planning of the 
Census and was then asked to come back for the taking of the Census, I can 
assure you the mid-decade hearings on how the census should be taken, even 
as difficult and sometimes contentious as they were, were extremely helpful in 
developing the final plans for the conduct of the Census. Later in this hearing you 
will hear from Robert Hill who participated in many of those hearings. His actions 
and that of many others led to unprecedented participation of leaders of minority 
organizations in the conduct of the 1980 Census. 

Given the responsibility of the Administration and the Legislature to ensure the 
proper conduct of the Census, I appreciate the need for and the importance of 
legislative hearings. The citizens have the right, through their elected 
representatives, to make sure things are planned for and conducted properly. 
Even at this late stage, I think hearings like this, where you are asking what have 
we learned that will be helpful in the actual conduct of the Census are vital. 
Further hearings on how to involve others in communicating the importance of 
participating in the Census would go a long way towards making the very difficult 
task facing the employees of the Census Bureau, in this particular Census, less 
difficult. 

Given where we are in the latter part of the decade, I would respectfully suggest, 
in the interest of getting the Census done well, that every effort be taken to 
conduct oversight involving the Census Bureau director in a manner that takes 
into account the extreme difficulty, the absolute deadlines, and the known 
problems which must be tackled and which will require his or her personal time to 
solve them. At this point the time has passed when new activities, beyond those 
already in place, can be planned for and implemented. 

Another lesson I learned, was how the personal integrity of the career employees 
of the Census Bureau served as an important check point on difficult questions 
that occur during the taking of the Census, which by its very nature is a political 
activity. In this context I am aware that concerns have been raised over the 
extent to which our new Administration will be involved in providing direction to 
the Census Bureau. 

My experience here is, in some ways, unique. When I served as Director during 
the planning of the 1 980 Census I came in as a registered Republican appointed 
by a Republican President. When I came back for the conduct of the Census I 
was still a registered Republican, but this time I was appointed by a Democratic 
President, In the conduct of the Census I had more direct contact with the White 
House than in the planning period. In fact, it was a condition of my return that 
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arrangements were made so that the Department of Commerce was aware that 
when it was necessary, because of timeliness required for an answer or the issue 
was outside of the Commerce area of interest, I would be able to work directly 
with the White house. 

In that time I can assure you that I was never asked to do anything that was not 
in the best interest of the Country. More importantly, if I had been asked to do 
something that, in my judgment, was not in the best interest of the Country, I 
would have been able to explain to that person making the request, that even if I 
had been willing to meet their request, they should be prepared to see that 
request on the front page of the Washington Post. The reason was simple; to get 
something accomplished the assistance of many Census Bureau career 
employees is required. Their feedback to a questionable action would be direct 
and if it was not listened to and properly addressed they would find a way to 
reveal the requested action rather than doing something that was not in the 
Country's best interest and that would eventually put the reputation of the 
Census Bureau at risk. 

Because of its dedicated employees the Census Bureau has a well-earned 
reputation for conducting a very political activity in a non-partisan way. 

There is one other observation I would make, in this case, to the judicial branch 
of government. On December 8, 1987, more than seven years following the start 
of the 1980 Census, (and after over fifty law suits were filed) United States 
District Judge, John E. Sprizzo decided that the appropriate standard for review 
was whether the Census Bureau had been either “arbitrary or capricious” in its 
decision not the adjust the 1980 census. Judge Sprizzo went on to say: 

“Indeed, the extensive testimony at trial overwhelmingly demonstrates that 
the determination as to whether the use of the currently available 
adjustment techniques will provide a more or less reliable estimate of the 
population than the unadjusted census is an extraordinarily technical one, 
about which reasonable statisticians and demographers can and do 
disagree. Certainly the Bureau, which has the necessary experience, 
expertise, and resources to collect and analyze the complex statistical 
data, is better equipped than the courts to decide whether, in view of this 
dispute among the experts, the census should be adjusted.” 

Being neither capricious nor arbitrary is at the heart of the Census Bureau 
culture. Those who disagree with the decisions made by the Census Bureau 
should determine whether to take action, only after determining whether the 
bureau was capricious or arbitrary in making that decision. Doing so will save the 
Country and the people they represent a lot of time and money. 

In the Carter Administration the decision, supported by Secretary of Commerce 
was that the Director of the Census Bureau should determine whether the 
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Census count should be adjusted. That decision should be maintained by the 
new Administration. 

Thank you for this opportunity to share with you a few of the lessons I learned 
while in a job which provided me a chance to make a contribution as well as the 
opportunity learn so much about our government and our people. 
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I am Barbara Everitt Bryant. I was Director of the Census Bureau froml989 to 
1993 and of the 1990 census. You have asked several of us to report on lessons learned 
from prior censuses and our views on conducting an accurate and cost-effective census. 

Lesson ,1: Timely Appointment of a Director 

My first recommendation to this Subcommittee is that you do everything in your 
power and use your influence on the Administration to get a new Census Director 
nominated and confirmed as soon as possible. The hour, one year before the census, is 
already late and on this I speak from experience. Twenty years ago I became Director 
very late in the same election cycle we are in now. That is, a Presidential election in the 
year ending in “8,” and nomination in the year ending in “9.” But I was not in office 
until December 7, three weeks before the census year was to start. I got in office only by 
a recess appointment when my administration — who had not sent my nomination to the 
Senate until early October-realized when Congress adjourned for the Thanksgiving-to- 
New Year recess that there would not be a Director in place for the Census, as my 
confirmation hearing had not been scheduled. I was eventually confirmed by the Senate 
in the summer after the count. 



39 


Would census procedures been different iff had been in office sooner? 

Definitely! With 25 years of survey research experience, I found the worst designed 
questionnaire I had ever seen already rolling off the presses. The Census Bureau had 
been so concerned about capturing the data, that they had designed a questiormaire that 
was data processing- friendly but not user-friendly. The graphics were poor, and the 
questiormaire was covered with marks used for registering cameras that transferred 
answers to microfilm, but may have made the questionnaire look complicated to the user. 
The lesson of good questionnaire design has now been thoroughly learned by the Census 
Bureau. In 1991 we brought in the national guru on mail questionnaire design. Dr. 

Donald Dillman, and began testing questionnaire versions. As a result, the 2000 census 
had an excellent questionnaire design. 

I also learned the difficulties of needing to immediately straighten out major 
problems when one is not yet acquainted with the organization’s personnel. To promote 
the census, the Census Bureau had produced many promotional materials — posters, 
brochures, pencils and other handouts. When I arrived in office in December I 
discovered these were sitting in the Census Bureau’s warehouse in Jeffersonville, 

Indiana — undistributed and influencing nobody. With the help of the Deputy Director — 
because I was at the disadvantage of not yet knowing census personnel and their talents — 
we shifted some personnel and put two excellent mid-level executives in place to take 
over distribution. For expediency, they had to do one-size-fits all shipments to the 496 
offices across the country set up for taking the census. This had some amusing side 
effects, such as “Alaska Natives Be Counted” posters being sent everywhere, including 
Puerto Rico, and posters designed for Indian communities by the Institute for American 
Indian Arts becoming well-loved art work on walls in every part of the country. 

However, the materials were moved in time to be used starting January 2 of the census 
year. 


I am a supporter of making the job of Director of the Census Bureau a 5-year 
appointment, starting in the years one and six. The planning cycle for operations as large 
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as the decennial and economic censuses are long and only with a several year lead-time 
could a Director have meaningful input to operations. Had I been in office earlier in 
1989, it would still have been too late to save the 1990 questionnaire. It is no fun for a 
Census Director to sit before Congressional committees, such as this one, defending 
operations in which he or she has had no input. 

Lesson 2: A major, professional, coordinated communications and advertising 
campaign is vital to census success and accuracy. Such a campaign requires major 
financial outlay. 

Such a campaign has two major components and their theme needs to be 
coordinated: 1) a large volume of inexpensive promotional materials that can be handed 
out at the local level; and 2) radio, TV, and newspaper spots professionally produced 
with goals of reaching both a mass national and targeted audiences. 

The advertising campaign has to be on a scale comparable to what a large private 
sector company would use to launch a new product. After all, the census is a new 
product to those in their 20’s. It is a 10-year old, half-forgotten product to those 30 and 
over. 


We learned in 1990 that the Census Bureau can no longer depend on free public 
service air time. Whereas that had worked in prior years, by 1 990 TV and radio stations 
were no longer required to give free air lime around the clock to keep their FCC licenses. 
Good and prime time space must be purchased, whatever the cost. 

Lesson 3: Outreach to Hard-to-Count Segments of the Population through 
Partnerships with Geographic, Ethnic, and Racial Organizations Can Help Reduce 
Undercount 

When we talk of undercount, let me put the proportions in perspective. There is 
very little overall undercount. The problem is differential undercount. The census 
historically has fully counted some segments of the population, such as homeowners and 
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older Americans. It falls short of fully counting the very mobile, renters, young people, 
and particularly those in Hispanic/Latino, African American, and American Indian 
communities. Those hard-to-count are best reached with one-on-one contacts from local 
people and organizations they know and trust. Communicating the fact that the Census 
Bureau will not give information from their census forms to any other organization or 
individual is a very hard message to get across. Only trusted sources can convince the 
reluctant, fearful, or uninformed that the Census Bureau does not give information to the 
INS, the IRS, landlords, ex-spouses, or mothers-in law. 

Factors That Will Help Improve Accuracy in 2010 

In addition to implementing these three lessons, three other factors will help 
improve accuracy in 2010: 1) The American Community Survey; 2) The downturn in 
employment that should make it possible to get better qualified temporary personnel; 3) 
The undercount research and coverage measurement done after recent censuses. 

1) The American Community Survey. The census will no longer send out the 
long form with nearly 50 questions to 1 7% of households. This has always had a 
response rate several percentage points lower than the short form with only the 7 or 8 
questions about each person needed for reapportionment and redistricting. The long form 
is now replaced by the American Community Survey which acquires the demographic, 
social and economic data formerly on the long form from a survey of one percent of 
households every year throughout the decade. 

2) The Downturn in Employment. In 1990 when the census was fielded, 
unemployment was at 5%. The pool of available non-employed persons had dried to a 
puddle. While the downturn in employment is a national tragedy, it does have the effect 
of making temporary census jobs look very good to those unemployed, and it will create 
a more qualified pool from which to draw. 
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3) Undercount Research. Post-census research measures coverage at a very 
detailed level geographically. It pinpoints both the level of coverage and the undercount. 
The Census Bureau makes good use of this research to target the hard-to-reach in the 
subsequent census because it knows where they are. Without good coverage 
measurement, the Census Bureau could not learn lessons from each census to improve the 
next one. 

The Big Inhibitor to a Good Count in 2010 

Fear is very apt to inhibit a full count in Hispanic/Latino and immigrant 
neighborhoods in 2010. The current Immigration and Naturalization Service (INS) raids 
on employers and neighborhoods to identify and deport undocumented immigrants is 
bound to make residents unwilling to be found or, if found, to give information to the 
government. Imagine if you were a census taker and went to the door of a household that 
housed both legal and undocumented persons. I flew in yesterday from Phoenix where 
some family households include both, and where there have been a number of recent 
raids to find and arrest the undocumented. Imagine that you introduce yourself and say, 
“I’m from the Census Bureau and I want to ask you a few questions.” What kind of 
cooperation do you think you will get? 

A Cost-Effective Census. 

Finally, with emphasis on counting every person and household the decennial 
census will never be a cheap operation. With its experience in the logistics of the 
operation and its magnitude, the Census Bureau probably does as cost-effective a job as 
any organization could. Current staff, not the Director of a $2.6 billion census conducted 
20 years ago, must report to you on present efforts to be cost effective. 

A large, and not predictable expense of each census is following up on non- 
responding households. This follow-up requires repeated and labor intensive calls. This 
expense could be greatly reduced if the Census Bureau could statistically sample the non- 
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respondents and estimate the rest, a method that I personally recommended after 1990.’ 
However, doing so was precluded by a bill sent by Congress to the Supreme Court before 
the 2000 census. The Supreme Court ruled against using sampling and estimation for 
apportioning Congressional seats. 


' Bryant, Barbara Everitt and William Dunn, Moving Power and Money: The Politics of Census Taking 
(Ithaca, New York: New Strategist Publications, Inc., 1995) 
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Mr. Chairman and members of the subcommittee, I am pleased to have the opportunity to 
speak to you regarding the 2010 census. The experience that I draw upon for my 
statement is based on a 27 year career at the Census Bureau. Throughout this period, I 
held a number of management positions with increasing responsibility for the 1980, 1990, 
and 2000 censuses, the most significant being from 1 997 through June 2001 when I 
served as the Associate Director for Decennial Censuses - the career executive with 
responsibility for all aspects of Census 2000. 

My discussion will first cover the major components that contributed to the success of 
Census 2000, and then will focus on how these may be useful in reducing the risks facing 
the 2010 census effort. 

However, before I begin, 1 would like to recognize the staff at the Census Bureau. They 
are highly motivated to carry out high quality non-partisan work. They are very creative 
and are the most significant source of the success of not only the decennial census, but 
the myriad of other demographic and economic programs the Census Bureau conducts. I 
know that for the 2010 census they are trying their best, and my remarks are intended 
only to assist their effort. 

Census 2000 Success factors 

Unprecedented support - Census 2000 received the support of a wide array of 
stakeholders at the local, city, state, tribal, and national levels that encouraged response 
and participation in the census. The Census 2000 final mail response rate of 67 percent 
exceeded the 65 percent figure for 1990, reversing a trend of declining census mail 
response. 

Three key factors led to this success, in my opinion. First, Census 2000 was the first 
census to use a paid advertising and promotion campaign. Second, an extensive 
partnership program directed at actively involving local communities was funded. Third, 
an effective communications strategy was established with key stakeholders to generate 
additional support for census participation. Stakeholders included the Congress, state, 
local and tribal governments, and numerous advocacy groups. 

In addition, the non-response follow-up contact of households that did not respond to the 
mail was completed in approximately 9 weeks, a first for census taking in the era of 
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censuses based on a strategy of mailing households a questionnaire to complete and a 
mail-back document prior to conducting an in-person follow-up for those households 
which did not respond via mail. There are several important factors underlying this 
achievement, including a high level of public motivation to respond to the census when 
an enumerator visited the households of those that did not respond to the mail. 

A Director of the Census Bureau was in place in time to provide leadership and 
direction during critical periods - Dr. Kenneth Prewitt was confirmed as Director in 
1998. He arrived in time to provide leadership and direction through several challenging 
periods. He was instrumental in establishing the communications strategy with the 
stakeholders mentioned above. In addition. Dr. Prewitt also put in place additional 
communication vehicles directed at providing timely and accurate information to the 
administration, the Congress, the Office of Management and Budget, other federal 
agencies, and a number of oversight bodies. The strategic vision and leadership was 
instrumental in the ultimate success of Census 2000, and created an environment where I 
and other career staff could be most effective. 

A strong experienced management team was in place well in advance of Census 2000 - 
The management team that led Census 2000 was very strong and experienced. In 
addition, much of the team was in place in time to make significant contributions to the 
planning effort. The team not only included persons with lengthy decennial experience, 
but was enriched by outstanding managers with different backgrounds. While my own 
experiences had been exclusively associated with the decennial census, other managers 
including the Deputy Director and the Principal Associate Director for Administration 
had come to the Census Bureau from other government agencies. This blend of 
experience created a synergy that led to creative thinking and problem solving from a 
very broad perspective. 

The Census 2000 field effort was well managed and well funded - The Field 
organization at the Census Bureau is led by a central office in Suitland, Maryland and by 
12 outstanding Regional Directors in offices across the United States. I simply cannot 
say enough about the efforts of this group for Census 2000. They were charged with 
recruiting and directing a staff that at its peak numbered close to 500,000. For 2000, 
early studies that linked enumerator pay to performance provided the information 
necessary to request higher relative pay rates than in previous censuses as a means of 
reducing turnover and overall costs. Once approved by the administration and Congress, 
these locality based pay rates gave the Field management group an additional tool to 
recruit a workforce that was highly motivated. 

As I mentioned above, the ultimate result was that the non-response follow-up operation 
was completed in a much shorter time frame than in prior censuses. In addition to the 
public outreach, two other factors contributed to this outcome. First, enumerator turnover 
was low (relative to previous censuses) and second, the management provided by the 
Field organization was outstanding. 
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Effective usage of private sector contractors - For Census 2000 a strategy was employed 
to rely on the private sector for expertise in advanced technology. Thus, contracts were 
awarded to private sector information technology companies to support some of the 
major census processing operations. The result was that the receipt, scanning, and 
processing of all of the census questionnaires was carried out by private sector 
contractors using optical scanning coupled with intelligent character recognition. Over 
80 percent of the handwritten questionnaire entries were captured electronically with 
extremely high accuracy rates. In addition, three large (200,000 square foot) temporary 
processing centers were set-up, staffed, and managed by private sector contractors. There 
was an associated significant effort within the Census Bureau to manage these large 
contracts. 

Thorough testing of all operational systems - The dress rehearsal for Census 2000 
occurred in 1998 and allowed for a final testing of all major operational systems, 
including the linkages between private contractor and internal Census Bureau 
developments. This became very important when the Bureau was forced to redesign the 
census processes in early 2009 as a result of the decision that sampling could not be used 
to produce results that would be used for apportionment. The redesign required the 
development and fielding of a number of new operations aimed at improving coverage. 
However, the core systems had been thoroughly tested, and protocols for testing system 
modifications were in place. The end result was that the new procedures that emerged 
from the redesign were incorporated and tested prior to fielding. In summary, we were as 
prepared as possible from a systems testing and monitoring perspective when the major 
operations started with the mail-out of questionnaires in March 2000. 

The 2010 Census Risks and Recommendations 

From my perspective, the risks for the 2010 Census fall into the following areas: 

Systems development and testing - The Census Bureau is very late in the development 
and testing program to support the non-response follow-up operation. The reasons for 
this are well documented - the Census Bureau will forgo the previous plan to use 
handheld computing devices and revert to a paper-based methodology. In addition, the 
operational control systems necessary to support this operation were shifted from a 
contractor to internal Census Bureau developers. The paper-based procedures and 
systems were simply not available in time to be used during the dress rehearsal. 

Therefore the dress rehearsal did not include a paper-based non-response follow-up 
operation. 

The situation is very serious since the non-response follow-up operation is the most 
expensive, labor intensive and critical operation that must be carried out for the 2010 
Census. The systems to control and manage this operation must provide information to 
assign work to and monitor the progress of over 500,000 enumerators. This information 
must be available for a number of levels of management including central office, regional 
office, local census office and crew leaders. In addition, the systems must closely link 
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with other key systems including data capture and cost and progress. A failure in these 
systems would be catastrophic, potentially causing a loss of control of the enumeration, 
costing hundreds of millions of dollars, and putting not only the accuracy but the ultimate 
delivery of the census at risk. 

The Census Bureau must conduct a complete test of these systems including assignment 
preparation, a significant in-person data collection operation (involving a local census 
office, crew leaders, and enumerators), transmittal and receipt of completed 
questionnaires, information exchanges with other key systems, and provision of 
management information. 

Director of the Census Bureau vacancy - Currently, there is no (to my knowledge) 
nominee to take over as the Director of the Census Bureau. It is critical that a Director be 
named as soon as possible to provide leadership and vision for not only the 2010 but the 
2020 census as well. One of the most important contributions that a Director makes is 
establishing lines of communication and credibility with stakeholders, advocacy groups, 
and oversight bodies. As stated above, one of the most important factors in the success of 
Census 2000 was the widespread support it received. The 2010 Census would benefit 
greatly from a similar outpouring of support. A well respected Director is a key necessity 
for such an outcome. 

Communications - Establishing strong lines of communication with the administration, 
the Office of Management and Budget, the Congress, oversight entities and the other 
state, local and tribal stakeholders is critical for a successful census. The Census Bureau 
has a good core staff in this area. However, an administration appointment to lead this 
effort is essential. 

Management staff - 1 know personally the senior management staff at the Census 
Bureau. They are all accomplished and extremely effective, and I am certainly not 
directing any criticism toward their performance. However, I believe that they are 
stretched very thin and could benefit from an infusion of additional personnel. One of the 
lessons that I learned from the build-up for Census 2000 was that there are excellent 
managers throughout the federal government who can not only step in to help 
immediately but can offer fresh perspectives. I encourage the administration to work 
with the Census Bureau to reach out to other agencies for additional management 
assistance. 

Coverage Measurement -The Census Coverage Measurement (CCM) program is an 
important tool in assessing the accuracy of the census. For the 20 1 0 Census the CCM is 
targeted at measuring the components of errors underlying the census process as well as 
providing measures of undercount (and over-count) for various population groups and 
areas. I believe that the current design falls short of meeting the goal of measuring 
component error. I also believe that the schedule of the CCM places key operations too 
far from Census Day, putting the entire program at risk due to respondent recall errors. I 
have communicated my concerns to the Census Bureau and I understand that some of 
them are being considered. 
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Experimental Program for 2020 - Dr. Brown will express concerns related to the Census 
Bureau’s planning for the 2020 Census. I fully endorse his statement. The 
administration and the Congress should challenge the Census Bureau to develop plans for 
a different kind of census in 2020. The Census Bureau has a long history of developing 
new methodologies to meet both challenges and opportunities that arise as our nation 
changes and grows. A new look now at the census for 2020 is very important in order to 
make use of the 2010 Census as a testing platform. 

Mr. Chairman and members of the subcommittee thank you for the opportunity to make a 
statement regarding the 2010 Census. 
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What GAO Found 

The decennial cer^us is an inherently fragile undertaking, requiring many 
moving parts to come together in a short time frame. For example, accurate 
cost esttmates help ensure that the Bureau has adequate funds, and that 
Congress, the administration, and the Bureau itself have reliable information 
on which to base advice and decisions. However, as GAO has reported before, 
the Bureau has insufficient policies and procedures mid inadequately trained 
staff for conducting high-quality cost estimation for the decennial census. 

A succe^ful census requires a complete and accurate address list. The Bureau 
sends thousands of census workers (listers) into the field to collect and verify 
address information, and this year for the first time, listers will use handheld 
compulere to collect data. During the dress rehearsal there were significant 
technical problems. A small-scale field test showed that these problems 
appear to have been addressed; however, the test was not carried out under 
full census-like conditions and did not validate all address canvassing 
requirements. 

Nonresponse follow-up, the Bureau’s largest and most costly field operation, 
was initially planned to be conducted using the handheld computers, but was 
recently changed to a paper-based system due to technology issues. The 
Bureau has not yet developed a road map for monitoring the development and 
implementation of nonresponse follow-up under the new design. Such a plan 
is essential to conducting a successful nonresponse follow-up. Furthermore, 
the system that manages the flow of work in field offices is not yet developed. 
Lacking plans for the development of both nonresponse follow-up and this 
management system, the Bureau faces the risk of not having them developed 
and fully tested in time for the 2010 Census. 

In an effort to reduce the undercount, the Bureau is implementing a program 
of paid advertising integrated with other communications strategies, such as 
partnerships with state, local, and tribal governments and community 
organizations. Moving toward 2010, the Bureau faces long-standing challenges 
with the nation’s linguistic diversity and privacy concerns, which can 
contribute to the undercounting of some groups. 

Since 2005, GAO has reported concerns with the Bureau’s management and 
testing of key IT systems. GAO is reviewing the status and plans for the testing 
of key 2010 Census systems, and while the Bureau has made progress in 
conducting systems, integration, and end-to-end testing, critical testing still 
remains to be performed before systems wUl be ready to support the 2010 
Census, and the planning for the testing needs much improvement. In short, 
while the Bureau has made some noteworthy progress in gearing up for the 
enumeration, with just over a year remaining until census day, uncertainties 
surround the Bureau’s overall readiness for 2010. 
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Mr. Chairman, Senator McCain, and Members of the Subcommittee: 

We are pleased to be here today to discuss with you the progress the U.S. 
Census Bureau (Bureau) has made in implementing the 2010 Census, and 
some of the critical challenges that lie ahead. We have been reviewing the 
national enumeration for decades on behalf of Congress. Over the years, 
through scoi^ of reports and testimorues, we have acquired broad 
institutional knowledge that gives us a historical view of lessons learned 
for implementing a successful census. As requested, our remarks today 
will focus on the current challenges the Bureau faces and how lessons 
learned from prior decennials can help produce a more cost-effective 
headcount. 

Perhaps the most impoifrmt lesson learned from past enumerations is that 
the census is large, complex, and its many procedures are interrelated, 
thus making it inherently fragile. An accurate population count requires 
the alignment of a myriad of factors that include the successful execution 
of dozens of census-taking operations, the public’s willingness to 
cooperate with enumerators, and the Bureau’s ability to effectively partner 
with thousands of state, local, and tribal governments, as well as 
community and other organizations. The bottom line is that while the 
census is under way, the tolerance for any breakdowns is quite small, In 
light of this difficult operational environment, effective stewardship of the 
Bureau is essential to help ensure the census stays on track and the 
agency continues to embrace a culture of performance and accountability. 
Key to this will be the timely appointment of a Census Director who is an 
efficient administrator, a respected technical professional, a strategic 
leader, and capable of working constructively with Congress, officials at 
all levels of government, as well as nongovernmental organizations and the 
statistical community. 

Other key valuable lessons learned include the importance of (1) sound 
risk management, (2) staying on schedule, (3) and conducting the census 
as a shared national undertaking involving Congress, government agencies 
at all levels, mrd the public at large. One or more of these lessons learned 
can be applied to the challenges the Bureau currentiy faces as it gears-up 
for the 2010 Census. 

As you know, the census has encountered several significant operational 
challei^es to d^e. Today is the first anniversary of when we first put the 
2010 Census on our high-risk list because of (1) long-standing weaknesses 
in the Bureau’s information technology (IT) acquisition and contract 
management function, (2) problems with the performance of handheld 
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computer used to collect data, and (3) uncertainty over the ultimate cost 
of the census, currently estimated at more than $14 billion.' In the past 
year, the Bureau has made progress on these challenges, in part by 
strengthening its risk management efforts. Still, the census remmns high 
risk because a critical risk management exercise planned for 2008 — a 
“dress rehearsal” of all census operations — was curtailed.^ As a result, key 
operations and systems, including some that will be used for the first time 
in a census, were not tested in concert with one another or under census- 
like conditions. 

This year, 2009, will be one of the most crucial time periods in this decade- 
long census cycle. The Bureau has already initiated large-block 
canvassing — ^an operation where temporary field workers validate address 
lists and maps for census blocks with more than 1,000 housing units in 
them. Next month, the Bureau is scheduled to conduct address canvassing 
for remaining census blocks when about 140,000 temporary employees 
will walk every known street in the country trying to update and verify the 
Bureau’s address list and maps for the country. Later in the year, in a 
separate effort, the Bureau is scheduled to update the locations of 
approximately 200,000 “group quarters" including homeless shelters, 
college residence halls, and group homes. The Bureau will also be opening 
hundreds of local census offices and refining plans for later operations. 

As requested, in our remarks today, we will discuss the state of the census, 
paying particular attention to die following: 

• the importance of reliable cost estimates and justifications for 
spending on census activities; 

• building a complete and accurate address list to know where to count 
people; 

• following up on missing and conflicting responses to ensure 
completeness and accuracy; 

• targeting communications and outreach efforts to reduce the 
differential undercount; and 

• designing, testing, and implementing technology to support the census. 


^GAO, Ivfonnalion Technology: Significant Problems of CriMccU Automation Program 
Contribute to Risks Facing SOW Census, G AO-0S-550T (Washington. D.C.; Mar. 5, 2008). 

^GAO, High-Risk Series: An Update, GAO-09-271 (Washington, D.C.: Jan. 2009). 
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Because the effectiveness of these activities will determine in large part 
the final cost and accuracy of the 2010 enumeration, they are important 
building blocks of a successful enumeration. 

Our testimony today is based on our ongoing and recently completed 
work. See the last page of this statement for a list of our recently issued 
census reports. To identify key issues the Bureau faces as it approaches 
the 2010 Census, we reviewed and analyzed scheduling, design, 
operational, and testing plans for the various census operations, data from 
the dress rehearsal sites, and documents related to the December 2008 
field test of the handheld computers in Fayetteville, North Carolina, and 
we interviewed Bureau staff. At the field test, we observed the handheld 
computers’ ability to collect and transmit address data by accompanying 
census workers as they went door-to-door. In February 2009, we also 
observed census workers conduct large-block canvassing using laptop 
computers. We conducted our work in accordance with generally 
accepted government auditing standards. Those standards require that we 
plan and perform the audits to obtain sufficient, appropriate evidence to 
provide a reasonable basis for our findings and conclusions based on our 
audit objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions. 

In summary, the Bureau has made commendable progress in rolling out 
key components of the census, making improvements to the handheld 
computers, certain risk management efforts, and how it will print the 80 
million maps needed by temporary field staff to carry out the enumeration. 
Nevertheless, at a time when planning activities should be reaching 
completion, m^or testing should be winding down, and there should be 
confidence in the functionality of census-taking activities, the Bureau 
instead finds itself lacking sufficient policies, procedures, and trained staff 
to develop high-quality cost estimates, and a number of operations and 
support systems still need to be designed, planned, or tested. In the 13 
months leading up to Census Day, the Bureau will be challenged to 
implement early operations, complete the final preparations for various 
activities, make refinements, and address any glitches that arise. With little 
time remaining, uncertainties surround the Bureau’s readiness for 2010. 


Background 


As you know, Mr. Chairman, the decennial census is a critical national 
effort mandated by the Constitution. Census data are used to apportion 
seats in toe Congress, redraw congressional districts, allocate billions of 
doUare in federal assistance to state and local governments, and for 
numerous other public and private sector purposes. 
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The Bureau estimates that the 2010 Census will cost more than $14 billion 
over its life-cycle, making It the most expensive census in our nation’s 
history. According to the Bureau, the increasing cost of the census is 
caused in part by various societal trends — such as increasing privacy 
concerns, more non-EInglish speakers, and people residing in makeshift 
and other nontraditional living arrangements — making it harder to find 
people and get them to participate in the census. 

In light of these challenges, it will be important for the Bureau to draw 
upon the lessons learned from previous decennials and apply them to the 
operational environment it faces today. 

Some broad lessons learned that we have identified from our past work 
that directly aftect the cost and accuracy of the census include the 
following: 

• Sound risk management is critical to a successful census as the risks to 
a cost-effective enumeration are interrelated, and a shortcoming in one 
operation could trigger subsequent activities to spiral downward. Of 
course the reverse is also true, where a success in one operation could 
have a number of positive effects later in the process. Rigorous up- 
front preparations, testing, and where feasible, contingency planning, 
are the best ways to stave off problems. Likewise, management 
information systems capable of tracking key operations with real-time 
measures are essential because they enable the Bureau to quickly 
address trouble spots. 

• It is important for the Bureau to stay on schedule, as the census is 
conducted against a backdrop of immutable deadlines, and an 
elaborate chain of interrelated pre- and post-Census Day activities are 
predicated upon those dates. Specifically, the Secretary of Commerce 
is legally required to (1) conduct the census on April 1 of the decennial 
year, (2) report the state population counts to the President for 
purposes of congression^ apportionment by December 31 of the 
decennial year, and (3) send population tabulations to the states for 
purposes of redistricting no later than April 1 of the year following 
Census Day. To meet these reporting requirements, it is absolutely 
critical for the Bureau to stay on schedule. The figure shows some 
planned dates for selected decennial events. 
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Figure 1: Timeline of Selected Decennial Events 
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Finally, the decennial census is a shared national undertaking, where 
Congress; other federal agencies; state, local, and tribal governments; 
nonprofit and private organizations; and, ultimately, the American public, 
all play vital roles in securing a complete and accurate population tally. 
Recognizing this, the Bureau fosters partnerships with these various 
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entities to help with such activities as recruiting census workers, boosting 
participation, and building the Bureau’s master address list. Mobilizing and 
coordinating these organizations requires an enormous effort on the 
Bureau’s part. 


Accurate c(»t estimates are essential to a successful census because they 
help ensure that the Bureau has adequate funds, and so that Congress, the 
administration, and the Bureau itself can have reliable information on 
which to base or advise decisions. However, as we have reported before, 
the Bureau has insufficient policies and procedures and inadequately 
trained staff for conducting high-quality cost estimation for the decennial 
census.® The Bureau does not have cost estimation guidance and 
procedures in place or staff that is certified in cost estimation techniques. 
The Bureau is developing a new budget management tool that will support 
the cost estimation process beyond 2010. As part of that effort, the Bureau 
will need to establish rigorous cost estimation policies and procedures and 
use skilled estimators to ensure that future cost estimates are reliable and 
of high quality. 

For example, to help manage the 2010 Census and contain costs, over 5 
years ago we recommended that the Bureau develop a comprehensive, 
integrated project plan for the 2010 Census that should include the 
itemized, estimated costs of each component and a sensitivity analysis^ 
and an explanation of significant changes in the assumptions on which 
these costs were based.® In response, the Bureau provided us with the 
2010 Census Operations and Systems Plan, dated August 2007. This plan 
represented an important step forward by including operational inputs and 
outputs and describing linkages among operations and systems. However, 
that document did not include itemized cost estimates of each component 
or sensitivity analyses, and thus did not provide a valid baseline or range 
of estimates for the Bureau and Congress. The Bureau has provided 
annual cost updates as part of its budget submission process, but these too 
have lacked cost analyses to support them. As the Bureau approaches the 


Providing Reliable 
Cost Estimates and 
Justifications for 
Spending as 2010 
Approaches Presents 
a Major Challenge for 
the Bureau 


■’GAO, 2010 Census: Censtis Bureau Should Take Action to Impiove the Credibility and 
Accuracy of Its Cost Estimate for the Decennial Census, GAO-Wi-554 (Washington, D.C.: 
Jun. 15, 2008). 

^Sensitivity analysis examines the effect of changing one assumption or cost driver at a 
time while holding all other variables constant. 

®GAO, 2010 Census: Cost and Design Issues Need to Be Addressed Soon, GAO-04-37 
(Washington, D.C.: Jan. 15. 2004). 
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final surge in the current decade-long decennial spending cycle, providing 
reliable c(^t estimates accompanied by sound justification, as we have 
recommended, will be important if Congress is to make informed 
decisions on the levels at which to fund the remainder of the 2010 
Decennial Census. 


Effective Address 
Canvassing Is 
Essential for a 
Complete and 
Accurate Count 


A complete and accurate list of all addresses where people live in the 
country is the cornerstone of a successful census because it identifies all 
households that are to receive a census questionnaire and serves as the 
control mechanism for following up with households that fail to respond. 
The Bureau goes to great lengths to develop a quality address list and 
maps, working with the U.S. Postal Service; federal agencies; state, local, 
and tribal governments; local planning organizations; private sector; and 
nongovernmental entities. For example, imder the Local Update of Census 
Addresses (LUCA) program, the Bureau is authorized to partner with state, 
local, and tribal governments, tapping into their knowledge of local 
populations and housing conditions in order to secure a more complete 
count.® Between November 2007 and March 2008, over 8,000 state, local, 
and tribal governments provided approximately 8 million address updates 
through the LUCA program. The Bureau will send thousands of temporary 
census workers, known as listers, into the field to collect and verify 
address information and update maps on-site, including verifying address 
updates provided through the LUCA program. 

Despite the Bureau’s efforts, an inherent challenge is locating 
unconventional and hidden housing units, such as converted basements 
and attics. For example, as shown in figure 2, what appears to be a small, 
single-family house could contain an apartment, as suggested by its two 
doorbells. TTie Bureau has trained listers to look for extra mailboxes, 
utility meters, and other signs of hidden housing units and is developing 
training guides for 2010 to help listers locate hidden housing. Nonetheless, 
decisions on what is a habitable dwelling are often difficult to make — what 
is habitable to one worker may seem uninhabitable to another. According 
to Bureau estimates, approximately 1.4 million housing units were missed 
in the 2000 Census. If an address is not in the Bureau’s address file, its 
residents are less likely to be included in the census. 


^Census Address List ImproTOment Act of 1994, Pub. L. No. 103-430. 
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Figure 2: Single or Multi-unit Housing? 
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Performance of Handheld 
Computers Have Improved 
in Field Testing, but More 
Information Is Needed to 
Evaluate Readiness for 
Address Canvassing 


A nationwide address canvassing operation for the 2010 Census is 
scheduled to begin this spring, when listers will use handheld computers 
for the first time to collect address data. Listers will add addresses that do 
not already appear on the Bureau’s list and mark for deletion any that they 
cannot verify according to the rules and guidance developed by the 
Bureau, 

When the handheld computers were tested during the dress rehearsal of 
the address canvassing operation, the devices experienced such problems 
as slow or inconsistent data transmission, freeze-ups, and difficullies 
collecting mapping coordinates.’ The software that managers used to 
review work productivity and assign work was also troublesome.® For 


’GAO, sow Census: Census Bureau’s Decision to Continue with Handheld Computers 
for Address Canvassing Makes Planning and Testing Critical, (JAO-08-036 (Washington, 
D.C.rJul. 31,2008). 

®GAO, SOW Census: Plans for D&xnnial Census Operations and Technology Have 
Progressed, But Muck Uncertainty Remains, GAO-0S-S80T (Washington, D.C.: Jun. 11, 
2008) 
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example, management reports were unreliable because they pulled data 
from incorrect source, and Bureau staff had difficulty using the work 
m^agement software to reassign work. 

The Bureau took steps to fix these issues, and in December 2008 
conducted a limited field test in Fayetteville, North Carolina, to test the 
functionality and us^ility of the handheld computer, including whether 
the handheld computer problems encountered earlier had been resolved. 
Although the Bureau’s final evaluation of the field test was due by the end 
of February 2009, we were not able to review it for this testimony. From 
observations of the December 2008 field test and interviews with Bureau 
officials, the Bureau appears to have addressed many of the handheld 
computer performance issues, as well as the problems with the work 
management software, observed during the dress rehearsal. This is an 
important and noteworthy development. 

Nonetheless, more information is needed to determine the Bureau's 
overall readiness for address canvassing as the field test was not an end- 
to-end systems evaluation, did not validate all address canvassing 
requirements, such as training and help desk support, and did not include 
urban areas. Additionally, the scale of the field test was a fraction of that 
of the address canvassing operation. The Bureau was to conduct a review 
of the readiness of the handheld computers in January 2009, but has not 
yet reported the results of that review. Finally, the Bureau's actual 
workload for address canvassing — about 144.7 million addresses — is 11 
million addresses more than the Bureau had planned for, leaving the 
Bureau with too few handheld computers to complete the workload in the 
time originally scheduled. In response, the Bureau will be extending the 
amount of time listers will be working in the field in affected areas, 
although not extending the end date of the operation, to compensate for 
the larger workload. 

During dress rehearsal, listers experienced problems using handheld 
computers when collecting address data for large blocks having more than 
1,000 housing units. According to the Bureau, the handheld computer did 
not have the capacity to efficiently collect data for large blocks. The 
Bureau has taken steps to mitigate this problem. In January 2009, the 
Bureau began using laptop computers and software already used in other 
operations to canvass the 2,086 blocks it identified as large blocks and by 
the end of February, the Bureau had completed approximately 80 percent 
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of large block canvassing.® In February 2009, we observed large-block 
canvassing in Atlanta, Georgia; Boston, Massachusetts; New York, New 
York; San Francisco, California; and Washington, D.C. Based on our 
preliminary observations, the laptops appear to work well and listers 
reported their training was satisfactory. We are in the process of 
discussing these and other observations with the Bureau. 


RllTPail Nppds to Bureau’s largest and most costly field operation is nonresponse 

follow-up. The Bureau estimates that it will employ over 600,000 
FinsllZG Field. D3.t3. temporary workers to collect data from about 47 million nonrespondening 

Collection Plans households over the course of 10 weeks in 2010. On April 3, 2008, the 

Bureau announced that it would no longer use handheld computers for 
nonresponse follow-up and would instead change to a paper-based 
nonresponse follow-up operation. According to the Bureau, this change 
added between $2.2 billion to $3 billion to the total cost of the census. 

In May 2008, the Bureau issued a plan that covered major components of 
the paper-based nonresponse follow-up. Bureau officials said ffiat they are 
developing a more detailed plan that would describe 2010 nonresponse 
follow-up operations and systems, workflow, m^or milestones, and roles 
and responsibilities of different census divisions. Although the plan was 
due in January 2009, it has yet to be completed. Because this plan serves 
as a road map for monitoring the development and implementation of 
nonresponse foDow-up, it will be important for the Bureau to complete 
this plan. 

The Bureau has changed plans for many aspects of nonresponse follow-up, 
and officials are determining which activities and interfaces will be tested 
and when this testing will occur. Although the Bureau has carried out a 
paper-based follow-up operation in past decennials, the 2010 Census 
includes new procedures and system interfaces that have not been tested 
xmder census-like conditions because they were dropped from the dress 
rehearsal. Bureau officials acknowledged the importance of testing new 
and modified nonresponse follow-up activities and system interfaces to 
reduce risk, but have not yet developed detailed testing plans. Given the 
number of tasks at hand and the increasingly shorter time frame in which 
to accomplish them, it will be important for the Bureau to monitor the 


^hese 2,086 large blocks are located in 332 counties and are concentrated in the following 
regions: Atlanta, Charlotte. Dallas, Denver, Los Angeles, and New York. 


GAO-09-430T 



61 


development of these testing plans, coordinate this testing with other 
activities, and ensure that testing occurs in time to take corrective actions, 
if needed. 

In our previous work, we have highlighted the importance of sound risk 
management in planning for the decennial census. The Bureau has 
strengthened aspects of its risk management process. For example, in July 
2008, the Bureau identified 31 nonresponse follow-up risks, such as lower 
than expected enumerator productivity. However, it has not developed 
mitigation plans for these risks. Officials said that they are reevaluating 
these risks and plan to develop mitigation plans for high- and medium- 
priority noru^ponse follow-up risks starting in spring 2009. However, the 
Bureau has not yet determined when these plans will be completed. 


Coverage Follow-up One of the Bureau’s long-standing challenges is resolving conflicting 

Operation Needs to Be information respondents provide on census forms. This problem can 

Finalized occur, for example, when the number of household members reported on 

a completed form differs from the number of persons for whom 
information is provided. In such instances, the Bureau attempts to 
reconcile the data during the coverage follow-up operation. For 2010, the 
Bureau plans to expand the scope of this operation and include two 
questions — known as coverage probes — on the census form to identify 
households where someone may have been missed or counted incorrectly 
(see fig. 3). 


‘®GAO, 201 0 Census: Plans for Decennial Census Operations and Technology Have 
Progressed, But Much Uncertainty Remains, G AC)-0S-S86T (Washington, D.C.; Jun. 1 1, 
2008). 
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Figure 3: Example of Coverage Probes from Draft 2010 Census Form 
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However, after testing the probes earlier in the decade, the Bureau found 
that one of the probes was problematic in identifying persons potentially 
missing from the count. Although these probes were included on the forms 
mailed out during the dress rehearsal, the coverage follow-up operation 
did not include cases from nonresponse foilow-up, which was canceled 
from the dress rehearsal. In the absence of a final test of the coverage 
probes in nonresponse follow-up, the effectiveness of the information 
generated by the probes is uncertain. 
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Fieldwork Management 
System for Most 
Operations Still Needs to 
Be Specified and 
Programmed 


A successful census depends, in large part, on the work carried out in the 
local census offices." For the 2010 Census, this field work cannot be 
accomplished without a properly functioning Operations Control System 
(OCS). This s^retem is intended to provide managers with essential real- 
time information, such as worker productivity and completion rates for 
field operations. It also allows managers to assign or reassign cases among 
workers. If the system does not work as intended, it could bog down or 
delay field operations and introduce errors into data collected. 


Initially, the Bureau had planned to use a contractor to develop OCS to 
manage the workflow for those operations relying on paper-based 
processes, such as group quarters enumeration and nonresponse follow- 
up. However, in August 2008, the Bureau created an internal program to 
develop OCS and other related infrastructure that are needed to support 
these operations. The Bureau is still in the process of developing OCS for 
paper-based operations. 


Although the Bureau has established a high-level schedule for testing of 
OCS, it has not yet finalized the requirements needed to begin its 
programming or developed a detailed schedule for conducting additional 
tests. Further, the Bureau has not yet fully defined how OCS will work 
together with other systems. Bureau officials said that the lack of detailed 
plans for operations, such as nonresponse follow-up, makes it difficult to 
finalize requirements for OCS or its testing plans. Our work on IT systems 
testing has shown that without adequate oversight and more 
comprehensive guidance, the Bureau cannot ensure that it is thoroughly 
testing its systems and properly prioritizing testing activities before the 
2010 Census. 


The Bureau Has Taken 
Steps to Improve Map 
Production, but Faces a 
Tight Schedule 


The Bureau estimates that it wiU need to produce approximately 30 million 
different map files from which 80 million paper maps will be printed to 
assist census workers in locating addresses in m^or census operations. 
The quality of maps and the timing of map printing are critical to the 
success of the census. In addition, many map production and printing 
activities must be conducted in sequence with no time to spare, putting at 
risk the Bureau’s ability to print its maps on time. The Bxu'eau has taken 


' ‘For all decennial census operations, the Bureau plans to hire 1.4 million temporary 
employees who will receive their training and work assignments through 494 local census 
offices, as well as the 12 regional census centers throughout tlie country. 
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positive steps to meet its requirements for map production and printing for 
20 10. For example, in June 2008, the Bureau decided to produce a generic 
map type in lieu of several operation-specific versions to reduce the 
number of map files to be produced. Furthermore, the Bureau is preparing 
to print most of its maps at the local census offices rather than at the 
regional offices, reducing the need to coordinate map delivery to the local 
census offices. In addition, the Bureau has replaced its labor-intensive 
quality assurance process with integrated, automated processes. These 
steps taken to improve workflow will become particularly important as 
the Bureau works to produce and print maps on an already compressed 
schedule. 

The Bureau’s schedule for producing and printing maps does not allow for 
any delays m receiving data from other operations or from the contractor 
delivering map files. For example, the Bureau intends to include map 
information from address canvassing, which ends in July 2009, in maps 
that will be used to validate locations of group quarters, wluch begins in 
September 2009. Bureau officials have stated that the turnaround time 
between these operations allows no slippage, and if these data are 
received late, an entire chain of subsequent map production steps would 
be thrown off schedule. Furthermore, according to the Bureau, local 
census offices need to receive map files from the contractor in time to 
print maps for certain field operations by January 8, 2010. However, the 
contractor is not scheduled to finish delivering the map files until 
January 19, 2010. Bureau officials said that they have taken steps to ensure 
that the necessary map files are delivered in time for printing but are still 
working to resolve the discrepancy. 


Census Marketing 
Programs Will Need to 
Improve Response 
Rates of Historically 
Undercounted Groups 


The Bureau goes to great lengths to reduce the undercount, especially 
among those groups likely to be undercounted at a higher rate than others, 
such as minorities and renters. For example, the Bureau plans to provide 
language assistance guides in 59 languages for completing the census, an 
increase from 49 languages in 2000. For the first time in 2010, the Bureau 
plans to send bilingual questionnaires to approximately 13 million 
households that are currently likely to need Spanish language assistance, 
as determined by analyzing recent data from a related Bureau survey 
program. 


The Bureau also plans to deploy a multifaceted commiinicatior^ campaign 
consisting of, among other efforts, paid advertising and the hiring of as 
many as 6^ partnership staff who will be tasked with reaching out to local 
governments, community groups, and other organizations in an effort to 
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secure a more complete count. Overall, the Bureau estimates it will spend 
around $410 million on Its communication efforts for the 2010 Census;. 
However, in constant 2010 dollars, this amount is somewhat less than the 
approximately $480 million that the Bureau spent marketing the 2000 
Census. 

Although the effects of the Bureau’s communication efforts are difficult to 
measure, the Bureau reported some positive results from its 2000 Census 
marketing efforts with respect to raising awareness of the census. For 
example, four population groups — non-Hispanic Blacks, non-Hispanic 
Whites, Asians, eind Native Hawaiians — indicated they were more likely to 
return the census form after the 2000 Census partnership and marketing 
program than before its onset. However, a Bureau evaluation 
demonstrated only a limited linkage between the partnership and 
marketing effort and improvements in actual census mail return behavior 
for these or other groups. Put another way, while the Bureau’s marketing 
activities might raise awareness of the census, a remaining challenge is 
converting that awareness into an actual response. Other marketing 
challenges include long-standing issues such as the nation’s linguistic 
diversity and privacy concerns, as well as a number of newly emerging 
concerns such as local campaigns against illegal immigration and a post- 
September 1 1 environment that could heighten some groups’ fears of 
government agencies. 


Managing and Testing 
of Information 
Technology Systems 
Remain a Concern 


Since 2005, we have reported on weaknesses in the Bureau’s management 
of its IT acquisitions, and we remain concerned about the Bureau’s IT 
management and testing of key 2010 Census systems. For example, in 
October 2007, we reported on the status of and plans for key 2010 Census 
IT acquisitions and whether the Bureau was adequately managing 
associated risks. “ We found critical weaknesses in the Bureau’s risk 
management practices, including those associated with risk identification, 
mitigation, and oversight We later presented multiple testimonies on the 
Bureau’s progress in addressing significant risks facing the 2010 Census. In 
particular, the Field Data Collection Automation (FDCA) program, which 
includes the development of handheld computers for the address 
canvassing operation and the systems, equipment, and infrastructure that 
field staff will use to collect data, has experienced significant problems. 


‘^GAO, Ivformaiion Tecfmology: Census Bureau Needs to Improve Its Risk Management 
of Decennial Systems, GAO-08-79 (Washington, D.C.: Oct. 5, 2007). 
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For example, in March 2008, we testified that the FDCA program was 
experiencing schedule delays and cost increases, and was contributing 
significant risk to the 2010 Census. At that time, we highlighted our 
previous recommendations to better manage FDCA and the other IT 
acquisitions. “ 

In response to our findings ^d recommendations, the Bureau has taken 
several steps to improve its management of FT for the 2010 Census. For 
example, the Bureau has sought external assessments of its activities from 
independent research organizations, implemented a new management 
structure and management processes and brought in experienced 
personnel to key positions, and improved several reporting processes and 
metrics. In part, due to our review of the FDCA program, the Bureau 
requested a revised cost proposal for the FDCA program which resulted in 
a cost reduction of about $318 million for the remaining 5-year life-cycle of 
the program. 

As we have previously reported, operational testing planned during the 
census dress rehearsal would take place without the full complement of 
systems and functionality that was originally planned, and it was unclear 
whether the Bureau was developing plans to test all interrelated systems 
and functionality. At your request, we reviewed the status and plans of 
testing of key 2010 Census systems. As stated in our report, which we are 
releasing today, we found that the Bureau has made progi'ess in 
conducting systems, integration, and end-to-end testing, but critical testing 
still remains to be performed before systems will be ready to support the 
2010 Census, and the pluming, execution, and monitoring of its testing 
needs much improvement.'^ We are making 10 recommendations for 
strengthening the Bureau’s testing of 2010 Census systems. Those 
recommendations address improvements needed in test planning, 
management, and monitoring. In response to our report, the Department 
of Commerce and Bureau stated they had no significant dis^eements 
with our recommendations. 


‘®GAO-08-550T. 

”GA0, Ir^formation Technology: Censtis Bureau Testing of SOW Decennial Systems Can 
Be Strengthened, GAO-09-262 (Washington, D.C.: Mar. 5, 2009), 
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Concluding 

Observations 


In summary, little more than a year remains until Census Day. At a time 
when m^or testing should be completed and there should be confidence 
in the functionality of key operations, the Bureau instead finds itself 
managing late design changes and developing testing plans. The Bureau 
has taken some important steps toward mitigating some of the challenges 
that it has faced to date, yet much remains uncertain, and the risks to a 
successful decennM census remain. 

Addressing these risks and challenges will be critical to the timely 
completion of a cost-effective census, and it will be essentia! for the 
Bureau to develop plans for testing systems and procedures not included 
in the dress rehearsal, and for Congress to monitor the Bureau’s progress. 

As always, we look forward to working with Congre^ in assessing the 
Bureau’s efforts to overcome these hurdles to a successful census, and 
providing regular updates on the rollout of the decennial in the critical 
months that lie ahead. 

Mr. Chairman and members of this Subcommittee, this concludes our 
statement. We would be happy to respond to any questions that you or 
members of the Subcommittee may have at this time. 

If you have any questions on matters discussed in this testimony, please 
contact Robert Goldenkoff at (202) 512-2757 or David A. Powner at (202) 
512-9286 or by e-mail at goldenkoffr@gao,gov or pownerd@gao.gov. Other 
key contributors to this testimony include Sher'rie Bacon, Thomas Beall, 
Steven Berke, Vyay D’Souza, Elizabeth Fan, Richard Hung, Andrea Levine, 
Signora May, Ty Mitchell, Catherine Myrick, Lisa Pearson, Kathleen 
Padulchick, Crystal Robinson, Melissa Schermerhom, Cynthia Scott, Karl 
Seifert, Jonadtan Ticchurst, Timothy Wexler, and Katherine Wulff. 
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Census Bureau Needs to Strengthen Testing of 2010 
Decennial Systems 


What GAO Found 

Although the Bureau has made progress in testing key decennial systems, 
critical testing activities remain to be performed before systems wiU be ready 
to support, the 2010 census. Bureau program offices have completed some 
testing of individual systems, but significant work still remains to be done, and 
many plans have not yet been developed (see table below). In its testing of 
system integration, the Bureau has not completed critical activities; it also 
lacks a master list of interfaces between systems and has not developed 
testing plans and schedules. Although the Bureau had originally planned what 
it refers to as a Dress Rehearsal, starting in 2006, to serve as a comprehensive 
end-to-end test of key operations and systems, significant problems were 
identified during testing. As a result, several key operations were removed 
from the Dress Rehearsal and did not undergo end-to-end testing. The Bureau 
has neither developed testing plans for these key operations, nor has it 
determined when such plans will be completed. 

Weaknesses in the Bureau’s testing progress and plans can be attributed in 
part to a lack of sufficient executive-level oversight and guidance, Bureau 
management does provide oversight of system testing activities, but the 
oversight activities are not sufficient. For example, Bureau reports do not 
provide comprehensive status information on progress in testing key systems 
and interfaces, and assessments of the overall status of testing for key 
operations are not based on quantitative metrics. Further, although the 
Bureau has issued general testing guidance, it is neither mandatory nor 
specific enough to ensure consistency in conducting system testing. Without 
adequate oversight and more comprehensive guidance, the Bureau cannot 
ensure that it is thoroughly testing its systems and properly prioritizing testing 
activities before the 2010 Decennial Census, posing the risk that these systems 
may not perform as planned. 


Status and Plans of 2010 System Testing 

System 

Testino status 

Testing plan 
comoieted 

Testing schedule 
comoieted 

Headouarters orocessina 

In orooress 

Partial 

Partial 

Master address and geographic 
mtormation 

in orooress 

Partial 

Partial 

DecenrHal resoonse Inteoratkm 

In orooress 

Partial 

Partial 

Field data collection automation 

In progress 

Partial 

Partial 

PaDsr^sed operations 


No 

Partial 

Data access and dis^mination 

In orooress 

Partial 

Partial 


Source GAO antfysis of Bureau Usta. 
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Mr. Chairman and Members of the Subcommittee: 

Thank you for the opportunity to participate in today’s hearing on the 2010 
census. The U.S. Census Bureau (Bureau) is relying on both the 
acquisition of new systems and the enhancement of existing legacy 
systems for conducting operations for the 2010 Decennial Census. As you 
know, the census is mandated by the U.S. Constitution and provides data 
that are vital to the nation. These data are used, for example, to 
reapportion and redistrict the seats of the U.S. House of Representatives, 
realign the boundaries of the legislative districts of each state, and allocate 
federal financial assistance. Carrying out the census is the responsibility of 
the Department of Commerce’s Census Bureau, which is relying on 
automation and technology to improve the coverage, accuracy, and 
efficiency of the 2010 census. Because the accuracy of the 2010 census 
depends in part on the proper functioning of these systems, both 
individually and when integrated, thorough testing of these systems before 
their actual use is critical to the success of the census. 

As you know, in March 2008, we designated the 2010 Decennial Census as 
a high-risk area, citing a number of long-standing and emerging 
challenges,' including weaknesses in the Bureau’s management of its 
information technology (IT) systems and operations. The 2010 Decennial 
Census remained as one of our high-risk areas in our recent high-risk 
update issued in January 2009.^ This statement summarizes the findings in 
our report, being released by the subcommittee today, on the status and 
plans of testing of key 2010 decennial IT systems.® 

Our work for this report was performed in accordance with generally 
accepted government auditing standards. Those standards require that we 
plan and perform the audit to obtain sufficient, appropriate evidence to 
provide a reasonable basis for our findings and conclusions based on our 
audit objective. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objective. 


'GAO, Information Technology: Significant Problems of Critical Automation Program 
Conlribule to Risks Facing 2010 Census, GAO-08-550T (Washington, D.C.; Mar. 5, 2008). 

%AO, HighrRisk Series: An Update, GAO-09-271 (Washington, D.C.; Jan. 22, 2009). 

®GAO, Information Technology: Census Bureau Testing of2010DecennialSystems Can 
Be Strengthened, GAO-0{i-262 (Washington, D.C.: Mar. 5, 2009). 
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Background 


The Bureau’s mission is to provide comprehensive data about the nation’s 
people and economy. The 2010 census enumerates the number and 
location of people on Census Day, which is April 1, 2010. However, census 
operations begin long before Census Day and continue afterward. For 
example, addre^ canvassing for the 2010 census will begin in April 2009, 
while the Secretary of Commerce must report tabulated census data to the 
President by December 31, 2010, and to state governors and legislatures by 
March 31, 2011. 

The decennial census is a major undertaking for the Bureau that includes 
the following major activities; 

Establishing where to count. This includes identifying and correcting 
addresses for all known living quarters in the United States (address 
canvassing) and validating addresses identified as potential group 
quarters, such as college residence hails and group homes (group quarters 
validation). 

Collecting and integrating respondent information. This includes 
delivering questionnaires to housing units by mail and other methods,* 
processing the returned questionnaires, and following up with 
nonrespondents through personal interviews (nonresponse follow-up). It 
also includes enumerating residents of group quarters (group quarters 
enumeration) and occupied transitional living quarters (enumeration of 
transitory locations), such as recreational vehicle parks, campgrounds, 
and hotels. It also includes a final check of housing unit status (field 
verification) where Bureau workere verify potential duplicate housing 
units identified during response processing. 

Providing census results. This includes tabulating and summarizing 
census data and disseminating the results to the public. 


Role of IT in the Decennial Automation and it are to play a critical role in the success of the 2010 
Census censusbysupportingdatacollection, analysis, and dissemination. Several 

systems will play a key role in the 2010 census. For example, enumeration 
“universes,” which serve as the basis for enumeration operations and 
response data collection, are organized by the Universe Control and 


*For example, in the “upditfe/Ieave" operation, after enumerators update addresses, they 
leave questionnmres at housing units; this occurs mainly m rural areas lacking street 
names, house numbers, or both. 
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Management (UC&M) ^stem, and response data are received and edited 
to help eliminate duplicate responses using the Response Processing 
System (RT^). Both UC&M and RPS are legacy systems that are 
collectively called the Headquarters Processing System. 

Geogr^hic information and support to aid the Bureau in establishing 
where to count U.S. citizens are provided by the Master Address 
File/Topolo^cally Integrated Geographic Encoding and Referencing 
(MAF/TIGER) sj^tem. The Decennial Response Integration System (DRIS) 
is to provide a i^stem for collecting and integrating census responses from 
all sources, including forms and telephone interviews. The Field Data 
Collection Automation (FDCA) program includes the development of 
handheld computers for the address canvassing operation and the 
systems, equipment, and infrastructure that field staff will use to collect 
(^ta. Paper-Based Operations (PBO) was established in August 2008 
primarily to handle certain operations that were originally part of FDCA. 
PBO includes IT systems and infrastructure needed to support the use of 
paper forms for operations such as group quarters enumeration activities, 
nonresponse follow-up activities, enumeration at transitory locations 
activities, and field verification activities. These activities were originally 
to be conducted using IT systems and infrastructure developed by the 
FDCA program. Finally, the Data Access and Dissemination System II 
(DADS n) is to replace legacy systems for tabulating and publicly 
disseminating data. 


Comprehensive Testing 
Improves Chances of a 
Successful Decennial 
Census 


As stated in our testing guide and the Institute of Electrical and 
Electronics Engineers (IEEE) standards,® complete and thorough testing is 
essential for providing reasonable assurance that new or modified IT 
systems will perform as intended. To be effective, testing should be 
planned and conducted in a structured and disciplined fashion that 
includes processes to control each incremental level of testing, including 
testing of individual systems, the integration of those systems, and testing 
to address all mlerrelated systems and functionality in an operational 
environment. 


®GAO, Year 2000 Computing Crisis: A Testing Guide, GA()/AJMD-10,i.21 (Washington, 
D.C.: Nov. 1, 1^) and IEEE Std. 12207-2008, Systems and Sc^tioare Engineering — 
Software Lifecycle Processes (Piscataway, N.J.: 2008). 
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Further, this testing should be planned and scheduled in a structured and 
disciplined fashion. Comprehensive testing that is effectively plumed and 
scheduled can provide the basis for identifying key tasks and requirements 
and better ensure that a system meets these specified requirements and 
functions as intended in an operational envirorunent. 


Dress Rehearsal Includes In preparation for the 2010 census, the Bureau planned what it refers to as 

Testing of Certain Systems Dress Rehearsal. The Dress Rehearsal includes systems and 
and Operations integration testing,® as well as end-to-end testing of key operations in a 

census-like environment During the Dress Rehearsal period, running from 
February 2006 through June 2009, the Bureau is developing and testing 
systems and operations, and it held a mock Census Day on May 1, 2008. 
The Dress Rehearsal activities, which are still under way, are a subset of 
the activities planned for the actual 2010 census and include testing of 
both IT and non-IT related functions, such as opening offices and hiring 
staff. 

The Dress Rehearsal identified significant technical problems during the 
address canvassing and group quarters validation operations. For 
example, during the Dress Rehearsal address canvassing operation, the 
Bureau encountered problems with the handheld computers, including 
slow and inconsistent data transmissions, the devices freezing up, and 
difficulties collecting mapping coordinates. As a result of the problems 
observed during the Dress Rehearsal, cost overruns and schedule slippage 
in the FDCA program, and other issues, the Bureau removed the planned 
testing of several key operations from the Dress Rehearsal and switched 
key operations, such as nonresponse follow-up, to paper-based processes 
instead of using the handheld computers as originally planned. 


’'Individu^ pn^ram offices manage individual system testing for the Dress Rehearsal, and 
integration testing is managed by the pairs of pro^am offices whose interfaces are being 
tested. 
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Bureau Is Making 
Progress in Key 
System Testing, but 
Lacks Plans and 
Schedules 


Through the Dress Rehearsal and other testing activities, the Bureau has 
completed key ^tem tests, but significant testing has yet to be done, and 
planning for this is not complete. Table 1 summarizes the status and plans 
for system testing. 


Table 1 : Status of System Testing and Plans 


2010 system testing 


System 

Dress Rehearsal system testing 

Testing status 

Testing plan 
completed 

Testing schedule 
completed 

Headquarters Processing--UC&M 
and RPS 

In progress 

In progress 

Partial 

Partial 

MAF/TIGER 

Completed 

In progress 

Partial 

Partial 

DRtS 

Completed 

In progress 

Partial’ 

Partial* 

FDCA 

Partially compieted" 

In progress 

Partial 

Partial 

PBO 

N/A‘ 

In progress 

No 

Partial 

DADS 

DADS* in progress 

DADS 11 in 

Partial 

Partial 


progress 

Source GAO analysis ol Bureau data 

‘Program officials staled that ORtS's test i^an and schedule were completed but will be modified to 
reflect changes resulting from the switch to paper-based operations. 

‘System testing related to operations removed from the Dress Rehearsal was not completed. These 
operations were later moved to PBO. 

‘The office to support PBO was created in August 2008 

‘DAOS system is being used for Dress Rehearsal system testing, but the replacement system. DADS 
II, is being developed and tested for 2010 operations. 


Bureau Has Conducted 
Limited Integration 
Testing, but Has Not 
Developed 2010 Test Plans 
and Schedules for 
Integration Testing 


Effective integration testing ensures that external interfaces work 
correctly and that the integrated systems meet specified requirements. 
This testing should be planned and scheduled in a disciplined fashion 
according to defined priorities. 

For the 2010 census, each program office is responsible for and has made 
progress in defining system interfaces and conducting integration testing, 
which includes testing of these interfaces. However, significant activities 
remain to be completed. For example, for systems such as PBO, interfaces 
have not been fully defined, and other interfaces have been defined but 
have not been tested. In addition, the Bureau has not established a master 


Page 5 


GAO-09-413T Information Technology 





75 


list of interfaces between key systems, or plans and schedules for 
integration testing of these interfaces. A master list of system interfaces is 
an important tool for ensuring that interfaces are tested appropriately 
and that the priorities for testing are set correctly. As of October 2008, the 
Bureau had begun efforts to update a master list it had developed in 2007, 
but it has not provided a date when this list will be completed. 

Without a completed master list, the Bureau cannot develop 
comprehensive plans and schedules for conducting systems integration 
testing that indicate how the testing of these interfaces will be prioritized. 
With the limited amount of time remaining before systems are needed for 
2010 operations, the lack of comprehensive plans and schedules increases 
the risk that the Bureau may not be able to adequately test system 
interfaces, and that interfaced systems may not work together as intended. 


Bureau Has Conducted 
Limited End-to-End 
Testing as Part of the 
Dress Rehearsal, but Has 
Not Developed Testing 
Plans for Critical 
Operations 


Although several critical operations underwent end-to-end testing in the 
Dress Rehearsal, others did not As of December 2008, the Bureau had not 
established testing plans or schedules for end-to-end testing of the key 
operations that were removed from the Dress Rehearsal, nor has it 
determined when these plans will be completed. These operations include 

update/leave, 

nonresponse follow-up, 


• enumeration of transitory locations, 


• group quarters enumeration, and 

• field verification. 


The decreasing time available for completing end-to-end testing increases 
the risk that testing of key operations will not take place before the 
required deadline. Bureau officials have acknowledged this risk in 
briefings to the Office of Management and Budget. However, as of January 
2009, the Bureau had not completed mitigation plans for this risk. 
According to the Bureau, the plans are still being reviewed by senior 
management. Without plans to mitigate the risks associated with limited 
end-to-end testing, the Bureau may not be able to respond effectively if 
systems do not perform as intended. 
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Bureau Lacks Sufficient As stated in o«r testing guide and IEEE standards, overeight of testing 
Executive-Level Oversight activities includes both planning and ongoing monitoring of testing 

and Guidance for Testing activities. Ongoing monitoring entails collecting and assessing status and 
progress reports to determine, for example, whether specific test activities 
are on schedule. In addition, comprehensive guidance should describe 
each level of toting and the types of test products expected. 

In response to prior recommendations, the Bureau took initial steps to 
enhance its programwide oversight; however, these steps have not been 
sufficient. For example, in June 2008, the Bureau established an inventory 
of ail testing activities specific to all key decennial operations. However, 
the inventory has not been updated since May 2008, and officials have no 
plans for further updates. 

In another effort to improve executive-level oversight, the Decennial 
Management Division began producing (as of July 2008) a weekly 
executive alert report and has established (as of October 2008) a 
dashboard and monthly reporting indicators. However, these products do 
not provide comprehensive status information on the progress of testing 
key systems and interfaces. Further, the assessment of testing progress 
has not been based on quantitative and specific metrics. The lack of 
quantitative and specific metrics to track progress limits the Bureau’s 
ability to accurately assess the status and progress of testing activities. In 
commenting on our draft report, the Bureau provided selected examples 
where they had begun to use more detailed metrics to track the progress 
of end-to-end testing activities. 

The Bureau also has weaknesses in its testing guidance. According to the 
Associate Director for the 2010 census, the Bureau did establish a policy 
strongly encouraging offices responsible for decennial systems to use best 
practices in software development and testing, as specified in level 2 of 
Carnegie Mellon’s Capability Maturity Model® Integration.’ However, 
beyond this general guidance, there is no mandatory or specific guidance 
on key testing activities such as criteria for each level or the type of test 
products expected. The lack of guidance has led to an ad hoc — and, at 
times — less than desirable approach to testing. 


^Capabilily Maturity Model* Integration is intended to provide guidance for improving an 
orgaruzation's processes and U»e ability to manage the development, acquisition, and 
maintenance of products and services. Hie model uses capability levels to assess process 
maturity. 
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Implementation of 
Recommendations 
Could Help Ensure 
Key Testing Activities 
are Completed 


In our report, we are making ten recommendations for improvements to 
the Bureau’s testing activities. Our recommendations include finalizing 
system requirements and completing development of test plans and 
schedules, establishing a master list of system interfaces, prioritizing and 
developing plans to test these interfaces, and establishing plans to test 
operations removed from the Dress Rehearsal. In addition, we are 
recommending that the Bureau improve its monitoring of testing progress 
and improve executive-level oversight of testing activities. 


In written comments on the report, the department had no significant 
disagreements with our recommendations. The department stated that its 
focus is on testing new software and systems, not legacy systems and 
operations used in previous censuses. However, the systems in place to 
conduct these operations have changed substantially and have not yet 
been fully tested in a census-like environment. Consistent with our 
recommendations, finalizing test plans and schedules and testing all 
systems as thoroughly as possible will help to ensure that decennial 
systems will work as intended. 


In summary, while the Bureau’s program offices have made progress in 
testing key decennial systems, much work remains to ensure that systems 
operate as intended for conducting an accurate and timely 2010 census. 
Tius work includes system, integration, and end-to-end testing activities. 
Given the rapidly approaching deadlines of the 2010 census, completing 
testing and establishing stronger executive-level oversight are critical to 
ensuring that systems perform as intended when they are needed. 

Mr, Chairman and members of the subcommittee, this concludes our 
statement. We would be pleased to respond to any questions that you or 
other members of the subcommittee may have at this time. 
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Statement 

Good afternoon Mr. Chairman and members of the Subcommittee. I am Lawrence 
Brown, professor of statistics at the Wharton School of Business and a member of the 
National Academy of Sciences. As such, I have been actively interested in issues relating 
to the decennial census for over a decade, especially since my testimony before 
Congressional subcommittees in 1997 and 1998. I have written two scholarly articles 
about the operation of the decennial census, and — perhaps more pertinently — have served 
on several National Academy of Sciences/National Research Council advisory panels 
involving census issues. This service includes the 1998-2004 Panel to Review the 2000 
Census and the 2005-2008 Panel on Coverage Measurement for the 2010 Census. I am 
currently the Chair of the Panel to Review the 2010 Census Program of Evaluations and 
Experiments. Many of my comments this afternoon are drawn from a very recent Letter 
Report of this Panel, mailed to Mr. Thomas Mesenbourg as Acting Director of the Census 
Bureau. 

As you know, the decennial census is a complex, expensive operation. It has been 
described as the nation’s largest peacetime mobilization. Research and development for 
the present, and looking to the future, are essentia! if this operation is to be completed 
now and subsequently in an accurate and acceptably economical manner. There are three 
issues I would like to bring to your attention from our panel’s interim report in 2008 and 
our current letter report to Mr. Mesenbourg. Two of these issues involve research and 
planning that should be part of the 2010 Census if the census is to remain a satisfactorily 
contemporary operation in 2020. The third concern is a more immediate one about 
research that should be conducted before fielding the 2010 Census. 

Before coming to the two issues of importance for the next decade. I’ll first 
address the more immediate concern. This concern arises as a consequence of the 
replacement of handheld computing devices for use in the nonresponse follow-up portion 
of the census. This operation — acronymically termed NRFU — is the single largest and 
most expensive part of the census, with over half a million census workers operating out 
of regional and local offices throughout the country to actively collect census data. 
Several members of your committee, as well as other members of Congress, and others in 
the government have already noted the unfortunate and considerable increase in census 
costs arising out of the failed effort to incorporate handheld devices into NRFU. I want to 
focus on a different aspect of this forced change in census plans. 

These devices and other equipment and software accompanying them had been 
designated to form the core of the Operating Control System. This is the system used to 
keep coordinated track of, and direct, daily operations for the army of field workers 
involved in NRFU. The Census Bureau is now in the process of restructuring the entire 
OCS. It has been reported that the new command control system will contain portions of 
the original design (to be supplied by the Harris Corporation, the original contractor), 
portions from other contractors, and components analogous to paper-based and computer- 
based systems used in the 2000 census. 

Given the complexity of conducting the decennial census, it has long been 
deemed essential to have a complete test “dress rehearsal” two years prior to the census 
so that flaws can be detected and corrected. However, the timing of the decision to revert 
from handheld computers to a paper-based NRFU process was such that the 2008 dress 
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rehearsal did not test NRFU - a major gap in pre-census testing. Because NRFU was not 
tested, the dress rehearsal also provided no information or testing on the interaction of 
NRFU processes with the redesigned coverage follow-up operation and other component 
census processes. 

The Census Bureau acknowledges that the dress rehearsal provided an inadequate 
test of the 2010 census processes. As a remedy, it has scheduled a number of small 
system and field tests of various components of the census OCS chain. However, given 
that the operational control system for the field data collection system will not be ready 
until the summer or fall of 2009, the Census Bureau has decided against a comprehensive 
test of the entire process due to the lack of time to design and carry out such a test. 

The panel believes that this testing strategy puts the Census Bureau in an 
extremely risky position. This stance leaves the Bureau vulnerable to technical flaws in 
the census process that involve interactions of the many components and subsystems; it 
also fails to detect any potential problems in the interaction between the system and 
census workers and administrators under field conditions. The Census Bureau needs to 
perform as full and realistic an operational test as feasible of all nonresponse follow-up 
systems from start to finish. 

It is true that ideally tests should be conducted in enough time to detect — and 
correct — any problems. But if time is too short to allow for a full cycle of test and 
correction, earlier detection of defects or inefficiencies can still be vital. Even if a flaw is 
discovered too late to be addressed in a pre-tested, systematic way, some contingency 
planning will likely be able to greatly reduce any negative consequences for the census 
itself 


Now let me turn to two research issues that concern the longer range future of the 
census. Our reports discuss two topics with strong potential effects on cost and quality — 
Internet data collection and the use of administrative records. Neither of these is 
scheduled to play a major role in the 2010 census. But 2010 is very nearby in terms of 
planning and implementing a large operation like the Census. So it seems much too late 
to fill these omissions in 2010. What concerns us now is the lack of effective plans to use 
the opportunity of the 2010 Census to conduct research. Such research should study how 
(and whether) these modem tools should be part of the 2020 Census. 

In the past two decades the Census Bureau has compiled an extensive database by 
matching various federal-record personal ID systems including Social Security files and 
IRS identity records. Limited research uses have been made of these data, including an 
experiment in the 2000 census that used an early version of the database as a case study 
comparison for two sites in Maryland and Colorado. However, administrative records 
such as these are not a major part of plans for the 2010 census or its program of 
evaluations and experiments, despite their substantial potential for both census cost 
reduction and quality improvements. 

Administrative records could be used to dramatically reduce the cost of 
nonresponse follow-up and improve the quality of the resulting data collected by 
avoiding inaccuracies in “last resort” enumerations (often supplied by proxy respondents, 
such as neighbors or landlords) and by providing higher quality information than is 
currently supplied by whole-person and whole-household imputation. In addition, 
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administrative records could be used to target the implementation of census processes. A 
key example is that administrative records could identify areas in which the Master 
Address File (MAF) is deficient, and therefore in need of an address canvass check prior 
to the decennial census. It is conceivable that this approach could dramatically reduce the 
costs of the currently 100 percent application of the address canvassing operation. Other 
potential uses of administrative records are also suggested in our reports. 

Although wide-scale use of administrative records to substitute for nonresponse 
follow-up in 2020 or thereafter would almost certainly require a change in legislation, the 
potential benefits of increased use of records in census processes should be studied in 
order to estimate the extent to which such changes would be economically and 
statistically desirable. Given that the use of administrative records in such a manner 
provides one of the few opportunities to substantially reduce census field costs in 2020, it 
deserves serious attention in the planned 2010 research, experiments and evaluations. 

The use of the Internet for data collection in the decennial census presents 
important opportunities for cost reductions and improvements in data quality. These 
include cost savings through the reduction in the number of forms that have to be scanned 
or keyed for data entry, reduction in the processing of requests for mailing of foreign 
language questionnaires, and savings in field work as a result of more prompt receipt of 
individual data. Use of the Internet may also yield quality improvements through easier 
access to foreign language questionnaires and online editing of census responses. 
Additionally, failure to allow the use of online response imposes the social cost of the 
Census Bureau’s appearing to be out of step with modem data collection and computing 
environments. 

We recognize that the basic steps to implement an Internet experiment in 2010 are 
nontrivial. However, the panel is confident that the challenges can be overcome, even 
within a tight time frame, as they were when the Census Bureau added a limited online 
response option in 2000. In addition, the Census Bureau’s own experience with Internet 
questionnaire development in the 2000 census can be tapped in the development of 
privacy safeguards, as can the experience of other countries in developing security 
protocols for online census response (including the 2006 Canadian census internet option 
which involved Lockheed Martin as a major contractor). 

I would like to close with a brief, general observation of my own. This 
observation is not based on our Panel’s current report, but is certainly not contradicted by 
anything in our reports. The Census Bureau needs an aggressive, well coordinated, 
forward looking research program. Such a program requires well-trained and creative 
personnel with broad authority to initiate research and the expectation of being allowed to 
follow through to recommend improvements and innovations. This research effort 
requires high ranking leadership that deserves and gets the attention and respect of the 
Director of the Census. There should thus be established a stable unit with a reasonable 
expectation of adequate and continuing funding throughout the decade, and this unit 
should be responsible for research and development of effective innovations for future 
decennial censuses and other Bureau surveys and programs. 

Thank you for the invitation to testify today. I would be happy to address any 
questions the Subcommittee might have. 
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THE NATIONAL ACADEMIES 

Advisers lo the Notion on Science, Engineering, and Medicine 


Panel on the Design of the 2010 Census Program of Experiments and Evaluations 

Committee on National Statistics 

Division of Behavioral and Social Sciences and Education 


500 Fifth Street NW 
Washington, DC 20001 
(202) 334-3096 
Fax: (202)334-3751 


February 19, 2009 


Mr. Thomas L. Mesenbourg 
Acting Director and Deputy Director 
U.S. Census Bureau 
4600 Silver Hill Road 
Washington, DC 20233 

Dear Director Mesenbourg: 

This letter relates to plans for tests and experiments planned for the 20 1 0 census. We 
write to call your attention to several time-sensitive concerns: (1 ) three crucial topics that should 
be included in the experimentation during the 2010 census, (2) testing plans preliminary to the 
census; (3) the retention of 2010 census data, and (4) the designs of the experiments currently 
planned for 2010. 

Background 

The Panel on the Design of the 2010 Census Program of Evaluations and Experiments 
(CPEX) has a broad charge: 

. . , [to] consider priorities for evaluation and experimentation in the 2010 
census. [The panel] will also consider the design and documentation of the 
Master Address File and operational databases to facilitate research and 
evaluation, the design of experiments to embed in the 20 1 0 census, the design 
of evaluations of the 2010 census processes, and what can be learned from the 
pre-2010 testing that was conducted in 2003-2006 to enhance the testing to be 
conducted in 2012-2016 to support census planning for 2020. Topic areas for 
research, evaluation, and testing that would come within the panel’s scope 
include questionnaire design, address updating, nonresponse follow-up, 
coverage follow-up, unduplication of housing units and residents, editing and 
imputation procedures, and other census operations. Evaluations of data 
quality would also be within scope. . . 

Pursuant to this charge, the panel transmitted an interim report providing general 
priorities for the CPEX program to the Census Bureau in late 2007 (National Research Council, 
2008) and plans to issue a final report in fall 2009, 

The panel met most recently on November 10-1 1, 2008. At that meeting. Census Bureau 
staff briefed the panel about the topics that it had chosen for inclusion in the 2010 CPEX 
program and presented the outlines of the designs for the experiments to be included in the 2010 
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census. On the basis of those briefings and subsequent discussion, and given the relatively late 
timing of our final report in the census experimentation planning cycle, the purpose of this letter 
is to continue to fulfill our charge by providing timely analysis and recommendations for the 
CPEX program. 

Experimentation During the 2010 Census: Missing Topics 

A key objective of our interim report (National Research Council, 2008) was to suggest 
priority topics for experimentation during the census. In particular, we urged that the topics 
chosen for experimentation have a direct bearing on visions for the 2020 census (however 
preliminary) so that they can serve as a first step for research in the intercensal period. We also 
explicitly recommended that the 2010 experiments be chosen to examine issues with the 
potential to achieve substantial cost reductions or important improvements in data quality in 
2020. 

In November 2008, the panel was informed that the Census Bureau has chosen topics for 
four experiments to be conducted during the 2010 decennial census: (1) a nonresponse follow-up 
contact strategy experiment, (2) a privacy notification experiment, (3) an alternative 
questionnaire experiment, and (4) a deadline messaging and compressed schedule experiment. 
We are deeply concerned that although the topics selected by the Bureau are of interest, they are 
not grounded in a vision for 2020, nor arc they directly linked to cost or data quality concerns. At 
the same time, we are concerned that two topics with strong potential effects on cost and quality 
and overall importance for 2020 that we discussed in our interim report are absent from the 
Bureau’s experimentation plans: Internet data collection and the use of administrative records. 
We reemphasize that these two areas of research are critically important. In addition, we believe 
that a very different alternative questionnaire experiment — one that tries multiple approaches to 
improve collection of census residence information — ^would be invaluable for the fiiture of 
census questionnaire design. 

Internet Experimentation The use of the Internet for data collection in the decennial 
census presents important opportunities for cost reductions and improvements in data quality. 
These include cost savings through the reduction in the number of forms that have to be scanned 
or keyed for data entry, reduction in the processing of requests for mailing of foreign language 
questiormaires, and savings in field work as a result of more prompt receipt of individual data. 
Use of the Internet may also yield quality improvements through easier access to foreign 
language questiormaires and online editing of census responses. Importantly, the use of online 
response would avoid the social cost of the Census Bureau’s appearing to be out of step with 
modem data collection and computing environments. 

An experiment in the 2010 census would provide a unique opportunity for examining the 
use of the Internet for deceimial census data collection. A key issue that needs to be explored in 
an experiment is how large a fraction of the population can be induced in a census environment 
to use the Internet as a response option, while not at the same time greatly increasing the 
possibility of disclosure or incurring other security problems. Therefore, we strongly recommend 
a 2010 census Internet response experiment to help determine ways to increase the likelihood of 
Internet response in 2020 and possibly also leam how to minimize any associated negative 
effects. This test should include a “push Internet” option as one of the experimental treatments 
whereby the initial mail contact strongly encourages Internet response, perhaps even by 
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excluding a paper questionnaire from that initial mailing. Such an experiment could also address 
the quality of the data collected through the Internet, including for those requiring foreign 
language questionnaires for whom the Internet may provide a convenient multi-language option. 

We recognize that the basic steps to implement an Internet experiment in 2010 are 
nontrivial: the design and testing of an online version of the census questionnaire, the 
development of protocols that protect census respondents from disclosure of information, and the 
integration of online returns with other census operations. However, the panel is confident that 
the challenges can be overcome, even within a tight time frame, as they were when the Census 
Bureau added a limited online response option in 2000. In addition to the Census Bureau’s own 
experience with Internet questionnaire development in the 2000 census, the experience of other 
countries in developing security protocols for online census response (including the 2006 
Canadian census) can be tapped as the Census Bureau develops privacy safeguards for online 
response in planning such an experiment. 

Use of Administrative Records Administrative records offer substantial potential for 
both census cost reduction and quality improvements. Administrative records could be used to 
dramatically reduce the cost of nonresponse follow-up and improve the quality of the resulting 
data collected by avoiding inaccuracies in “last resort” enumerations (often supplied by proxy 
respondents, such as neighbors or landlords) and by providing higher quality information than is 
currently supplied by whole-person and whole-household imputation. (An admittedly radical 
eventual possibility for the use of administrative records would be avoidance of nonresponse 
follow-up altogether for a large percentage of U.S. households.) 

In addition, administrative records could be used to target the implementation of census 
processes. A key example is that administrative records could identify areas in which the Master 
Address File (MAF) is deficient, by basing that determination on the difference between the 
address counts from a merged list of addresses from administrative records and the counts from 
the MAF, and therefore in need of an address canvass check prior to the decennial census. This 
approach could dramatically reduce the costs of the currently 100 percent application of the 
address canvassing operation. One could also use the discrepancy between a household count 
from the census and that from administrative records to prioritize the implementation of 
coverage follow-up interviews. Finally, administrative records could be used to assist in 
reducing the field work in following up nonmatching cases of the P-sample in coverage 
measurement. 

Although wide-scale use of administrative records to substitute for nonresponse follow- 
up would almost certainly require a change in legislation, the potential benefits of increased use 
of records in census processes should be studied in order to estimate the extent to which such 
changes would be economically and statistically desirable. Given that the use of administrative 
records in such a manner provides one of the few opportunities to substantially reduce census 
field costs in 2020, it deserves serious attention in the planned 2010 experiments. 

It is important to note that most of the above possibilities for research on administrative 
records might be properly considered priorities for “evaluation” rather than “experimentation” 
since they would not require additional or special field data collection. (They would, however, 
require the careful retention of household-level census process data, such as we recommend 
below.) Yet although a great deal about the utility of administrative records can be learned from 
post hoc study of data retained during the census, there are potentially useful possibilities for 
limited, experimental field work in 2010. For instance, with regard to the use of administrative 
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records as a substitute for late-stage field enumeration, one possible experiment would involve 
variations in nonresponse follow-up or coverage follow-up protocols under which the number or 
format of follow-up interviews depended on administrative records information (either on an 
individual household basis or on an area basis). Such an experiment would involve a significant 
expansion of the nonresponse follow-up contact strategy experiment (discussed below). 

Though “administrative records” in the census context are generally thought to be 
national-level constructs — drawing information from, for example, Social Security 
Administration registers — a complete evaluation of records-based methods should also assess the 
quality of the records maintained by “group quarters” facilities, such as prisons, health care 
facilities, and college residence halls. Because these facility records were used by census 
enumerators to count about half of the group quarters’ population in the 2000 census, the 
National Research Council (2006:Table 7-1, pp. 238-240) suggested that the Census Bureau 
“undertake a continuing research effort to assess the accessibility of facility records at group 
quarters facilities and to determine whether the existing data systems meet census data collection 
needs.” We endorse this suggestion as it is an essential step to assessing the possibilities for 
using administrative records to supplement or, as necessary, replace traditional enumeration in 
group quarters. Assessing the alternative or “home” address information available from facility 
records is also critical to addressing such long-standing questions as the degree to which college 
students are counted at both their schools and their parental homes and whether it is feasible to 
define a “home address” for persons under correctional supervision. 

Census Residence The 2010 census provides a uniquely valuable setting for a 
comprehensive experiment involving alternative approaches to the current residence rules. The 
Census Bureau’s proposed alternative questionnaire experiment for 2010 does include one 
treatment group for gathering a limited amount of information on residence (see below). 
However, given that unclear residence mles and interpretations were likely a major source of 
census coverage error (both omission and duplication) in the 2000 census (National Research 
Council, 2004), the Panel on Residence Rules in the Decennial Census (National Research 
Council, 2006) suggested various alternative approaches to collecting information on census 
residence. In particular, that panel’s report proposed a major change from the Census Bureau’s 
traditional approach of relying on a dense set of instructions at the start of the census form to one 
of asking a set of guided questions that breaks the large cognitive task of deciding one’s 
household composition into smaller pieces. At that panel’s urging, the Census Bureau tested a 
preliminary version of a “worksheet” approach to the residence question in 2005, yet no further 
work on residence is planned in 20 1 0. 

The single treatment group in the proposed alternative questionnaire experiment — 
anchored to one of the coverage probe questions — falls short of the general “any residence 
elsewhere” query that the National Research Council (2006) recommended be asked of the 
general population in a 2010 census experiment and asked of all group quarters (e.g., medical 
facilities and college housing) residents in the 2010 census itself The current plans for this 
limited experiment also do not appear to include the follow-up activities needed to make best use 
of whatever information might be gained. The proposed single treatment group also falls short 
of the 2006 report’s suggestion to experiment with a de facto or “current residence” question — 
and add a corresponding de jure or “usual residence” question to the American Community 
Survey — so that differences in estimates between the two programs due to their differing 
residence standards could be assessed. Innovative (and more accurate) handling of residence 
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concepts is clearly a research question for which several alternatives need to be tested, and 
subsequently refined and retested, in order to achieve substantial gains over the Bureau’s current 
approaches. 

These three research areas — Internet data collection, the use of administrative records, 
and questionnaire redesign for residence rules — are ones for which important benefits could be 
obtained through increases in census data quality or decreases in census costs or both. In the 
panel’s assessment, the 2010 CPEX program should include work on these topics in order to 
ensure early progress in the 2020 census testing cycle. Therefore, we strongly urge that these 
topics be included as subjects for experiments in conjunction with the 2010 census. 

Systems Testing and Simulation Prior to the 2010 Census 

The panel is concerned that the Census Bureau’s operational test plans for the 2010 
census are insufficient. We are particularly concerned with the Bureau’s capacity to identify 
potential failure modes in the field data collection components of the 2010 census process. We 
appreciate that the Census Bureau has had to substantially revise its plans for decennial census 
nonresponse follow-up. Initial plans to use handheld computers for nonresponse follow-up and 
to have the operational control system for field data collection developed by a contractor have 
been dropped in favor of a return to a paper-based nonresponse follow-up operation and a return 
to an operational control system for field data collection that will be developed in house 
(presumably by revising the system developed for the 2000 census). 

Given the complexity of conducting the decennial census, it has long been deemed 
essential to have a complete test “dress rehearsal” two years prior to the census so that flaws can 
be detected and corrected. Given the need to redesign the field data collection plan at this late 
stage, the census dress rehearsal conducted in 2008 was essentially limited to a test of the 
mailout/mailback portion of the census process, with no testing of the nonresponse follow-up, 
coverage follow-up operations, or many other component processes. 

The Census Bureau acknowledges that the dress rehearsal provided an inadequate test of 
the 2010 census processes. As a remedy, it has scheduled a number of small field tests of 
various components and sub-systems of the census process chain to attempt to identify as many 
potential flaws as possible prior to implementation. However, given that the operational control 
system for the field data collection system will not be ready until the summer or fall of 2009, the 
Census Bureau has decided against a comprehensive test of the entire field data collection 
process due to the lack of time to design and carry out such a test. 

The panel believes that this testing strategy puts the Census Bureau in an extremely risky 
position should there be flaws in the census process that involve interactions of the many 
components and subsystems. Testing the interfaces between individual components of a system 
(e.g., A — *5, B-*C, C-^D) can produce useful information and detect unseen problems. But the 
Bureau’s testing plan creates risks by not adequately testing subsystems (e.g., A or 

complete systems. Errors at this level may not be evident in any single component test but could 
result in major delays and impair data quality. 

Concern over the lack of time or resources to conduct a more comprehensive test is 
understandable, but it does not override the compelling argument for carrying out such a test. 
The Census Bureau needs to perform as full and realistic an operational test of all nonresponse 
follow-up systems as possible. The consequences of failure to identify substantial problems in 
the interfaces between system components could be dire, ranging from moderate to severe 
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impacts on the quality, costs, and timeliness of census counts for important purposes like 
redistricting and allocation of funds. 

The panel strongly recommends that the Census Bureau try to fit into its schedule a 
comprehensive test of the entire operational control system for field data collection as soon as 
feasible after plans for this system become available. We recognize the enormous constraints in 
planning and accomplishing such testing. Because of these constraints, it may well be necessary 
in the overall testing to simulate portions of the process based on the specifications for 
information flows at the interface between component parts of the process. If such simulation is 
judged to be necessary, then additional field testing of the simulated components of nonresponse 
follow-up should be carried out. 

Ideally, tests should be conducted in enough time to detect — and correct — any problems. 
But if time is too short to allow for a full cycle of test and correction, earlier detection of defects 
or inefficiencies can still be vital. Even if a flaw is discovered too late to be addressed in a pre- 
tested, systematic way, some contingency planning will likely be able to greatly reduce any 
negative consequences for the census itself. 

Retention of Data 

Since 1985 several National Research Council panels on the decennial census have called 
for the development of a “master trace sample” database. Such a database would retain the 
crucial elements of the census procedural history for a sample of addresses to support census 
evaluation studies. A version of a master trace sample was constructed by the Census Bureau 
following the 2000 census (Hill and Machowski, 2003). This database supported a small number 
of studies (e.g., Bentley and Tancredo, 2005; Tancredo and Bentley, 2005; West et al., 2005) that 
began to realize some of the substantial research potential that such a database could provide. 

Our panel’s interim report recommended that “the Census Bureau should initiate efforts 
now for planning the general design of a master trace sample database and should plan for 
retention of the necessary information to support its creation” (National Research Council, 
2008;Rec. 5). To address the efficacy of less common procedures on small subpopulations, a 
large sample is clearly needed; we also note that given the greatly decreased cost of computer 
storage and memory, it may now be possible to save and efficiently access the entire procedural 
history for the entire country. Whatever the sampling rate, it is critical to retain sufficient data, 
preserving all relevant linkages, so that the result supports the examination of how the decennial 
census processes functioned for various subpopulations and domains. 

As an example, it is important to retain the information as to which addresses on the 
MAP were added or deleted by which census address improvement operations. Furthermore, 
given that many fields of the various system files are overwritten continuously during the census, 
this means that these data archives should retain snapshots of files that will change during the 
course of census operations, and this should be provided for as frequently as needed. This data 
archival effort needs to include all parts of the census process, including address list 
development, nonresponse follow-up, coverage follow-up, group quarters enumeration, data 
capture and data treatment, and coverage measurement. In addition, it is vital that the schema 
used in retaining these data be carefully documented so that it is known precisely what is saved 
in each data field. 

Given the rushed development of the operational control system for field data collection, 
we are especially concerned that provisions be made for retaining data relating to that part of the 
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census. We do not believe that providing for this additional functionality in the operational 
control system for the field data collection will add appreciably to the current challenge of 
developing such a system in time for the 2010 census. Furthermore, by guaranteeing access to 
this information, the Census Bureau would ensure that it could carry out evaluations that would 
guide the Bureau towards a more effective and cost-efficient design for the 2020 census. 
Therefore, we recommend that — as systems for the 2010 census are finalized by the Census 
Bureau and its contractors — appropriate archival outlets be created for all systems, including 
components of the field data operational control system, so that the relevant data to construct a 
master trace database or “audit trail” of census processes are retained. Experts in automated audit 
processes could provide assistance to the Census Bureau in implementing a master trace system. 

Designs for Currently Planned Experiments 

Although we recommend the addition of three topics for experimentation, the Census 
Bureau’s chosen topics for 2010 experiments do concern issues that may be worth pursuing in 
addition to our recommended ones. However, three of the four census experiments, as currently 
outlined, suffer from important defects that will limit their effectiveness. Moreover, the Bureau 
has not carried out explicit studies of the statistical power of these experiments given their 
proposed designs. We recognize that the clustering inherent in some of the experimental designs 
complicates the development of such estimates, but it is also the reason that careful estimates of 
power are necessary. For each experiment, the Census Bureau needs to undertake a study of the 
statistical power of the design against reasonable alternatives based on anticipated effect sizes. 
This should be done not only for national-level comparisons, but also for any relevant subgroup 
comparisons. 

Some of the experiments also do not seem to give appropriate attention to “targeting” or 
oversampling respondents from relevant sociodemographic groups (or geographic areas with 
large concentrations of such respondents). Not only does lack of targeting reduce the power of 
those experiments, but it also hinders the ability to learn more about the response by stratifying 
the analysis by subgroup. 

The Nonresponse Follow-up Contact Strategy Experiment The question of interest in 
this experiment is the impact on census costs and data quality of reducing the number of attempts 
made in nonresponse follow-up from a maximum of six to either four or five. As currently 
planned, the experiment will be carried out in three local census offices, comprising about 
40,000 housing units. For each office, two treatments and the control will be randomly allocated 
to crew leader districts, where all enumerators in a district will use the same questionnaire 
(which provides space for a maximal number of enumeration attempts) but will receive different 
instructions about how many callbacks to make. To assess the treatments and control, 
comparisons will be made of the resulting impact on census data quality, measured by the rate of 
proxy response, the distribution of response outcomes, the item nonresponse rate, and measures 
of fomi completeness. The Census Bureau staff have expressed a concern as to whether the 
findings would be generalizable from the three local census offices, and asked the panel for 
assistance in selecting local census offices for this experiment. However, our current overriding 
concern is whether data from only three local office areas can ever be sufficiently generalizable. 

In addition to questions about generalizability and statistical power, the panel questions 
whether the likely reduction in field data collection costs will be sufficient to justify the 
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allocation of resources for an experiment during the 2010 census. The likely impact on census 
costs might be fairly modest. In the November meeting, the panel suggested that the cost 
reduction could be estimated on the basis of the frequency of enumerations in 2000 that were 
successful on the fourth or fifth attempts. The Census Bureau argued that such estimates are 
misleading due to infrastructure changes that occur during the taking of the census, such as the 
laying off of enumerators, consolidation of work, and other changes. The panel countered that 
estimates based on an analysis of 2000 census data, while somewhat flawed due to such changes, 
would still provide a sense of whether the potential reductions in field costs would be large 
enough to justify a separate experiment during the 2010 census. Based on such estimates, if the 
cost reduction seems likely to be, at best, modest, the experiment should be eliminated or 
redesigned to include assessment of even fewer enumeration attempts or the use of 
administrative records in lieu of field data collection. 

In considering statistical power, 2000 data could have been used to estimate the 
percentage of housing units that first failed to return their mailed questionnaire, and then were 
enumerated in the 2000 census on either the fourth or fifth attempt during nonresponse follow- 
up. In doing so, it may be discovered that the effective sample size for this experiment is too 
small to provide sufficient power to identify important differences in the above data quality 
measures (unless such differences are strikingly large). If it is clear that the experiment will not 
have substantial power to detect reasonable changes to the census data quality measures, and if a 
two or three-fold increase in the number of local census offices would provide sufficient power, 
the sample size should be expanded. If no conceivable sample size can provide reasonable 
statistical power, the experiment would not be useful and should not be done. 

One additional argument in favor of an experiment on this topic, if slightly broadened, is 
that there is an a distinct disadvantage of waiting until six responses are attempted. This 
disadvantage is that the lag between Census Day and the day of enumeration increases the 
number of movers and in general reduees data quality and increases the rate of erroneous 
enumeration. Assessment of this disadvantage, possibly in conjunction with the coverage 
measurement program, might be very useful. 


The Privacy Notifleation Experiment The privacy notification experiment will assess 
the effect of a message on the cover letter of the mailing package containing the census 
questionnaire regarding the uses of census data and the possible use of administrative records. 
The experiment includes two panels of 10,000 sampled households each (plus a control group 
without such notification), chosen using strata based on levels of mail response in the 2000 
census or in the American Community Survey. The assessment of the three wordings will use 
response rates, data quality measures, and monitoring of public reaction. The hope is to be able 
to have reasonable power to identify a difference in overall mailback rate of 1 .8 percent. (A one 
percent reduction in mail response is estimated to cost the Census Bureau $90 million in 2010.) 

The panel has three principal concerns with the current design of this experiment. The 
treatment panels vary only in the wording of one part of the notification message — “Your 
answers will be used for statistical purposes, and no other purpose” compared with “Your 
answers will only be used to produce statistics” — raising concerns about how informative the test 
will actually be regarding individual perceptions of privacy. Second, a longer, second section of 
the message is identical between the two treatment groups and hints at the possible use of 
administrative records: 
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To improve census results, other government agencies may give us information about 

your household. The additional information we receive is legally protected under Title 

13, like your census answers. 

If the objective of the experiment is to assess privacy concerns, it would be beneficial to explore 
other wordings of this second part of the notification. Instead of a single test of a very limited set 
of alternative statements in 2010, it would be more useful for the Census Bureau to conduct a 
series of intercensal tests between 2010 and 2020 that would develop a broad sense of people’s 
sensitivity to privacy concerns and use of administrative records. Such a research program 
should examine this for sociodemographic subsets of the population. 

Another deficiency is that the Census Bureau is not using this opportunity to evaluate the 
implied tradeoff of the costs incurred from the freedom to use administrative records as a result 
of the inclusion of such a notification and the benefits from being allowed to do so. That is, 
while the privacy notification may have the effect of reducing mail response rates, it will at the 
same time allow for the use of administrative records to reduce costs and improve data quality, 
for example, by substituting for last-resort and proxy enumeration. Therefore, it seems 
reasonable to use this opportunity to determine the degree to which administrative records can 
reduce census costs and improve census data quality and whether such benefits offset the 
reduction in mail response and the associated increase in the costs of nonresponse follow-up. 
Possibly, this could be done through the separate administrative records experiment noted above, 
but bundling this as a single experiment may have some advantages, although it would increase 
the complexity of the currently planned experiment. 

The Alternative Questionnaire Experiment There are three parts to the proposed 2010 
questionnaire experiment: (a) a comparison of the complete set of questionnaire changes between 
2000 and 2010, (b) an attempt to collect an alternative residence address based on answers to a 
coverage probe question, and (c) alternative formats for the collection of information on race and 
ethnicity. In part (a), 10,000 housing units will receive a 2000-style census questionnaire. 
Comparisons will be made to the distribution of responses to the full 2010 census to ascertain 
what changes between 2000 and 2010 are due to changes in questionnaire format. In part (b), 
30,000 housing units will be administrated an alternative questionnaire that will permit 
respondents to specify a street address if they indicate that the person in question sometimes 
lives or stays at another location. In part (c), 30,000 housing units in each of 1 1 panels will be 
administered various questionnaire formats for the questions on race and ethnicity. Some of 
these will present slightly different versions of a combined race and Hispanic origin question (the 
2010 census questionnaire itself presents them as separate numbered items). Other treatment 
groups respond to census advisory committee suggestions by permitting multiple and write-in 
answers to the Hispanic origin question or varying specific examples that are explicitly 
mentioned in the question (e.g., Taiwanese or Marshallese), It is planned that cognitive testing 
will be carried out in advance of the experiment to better refine the various alternatives. The 
forms will be mailed to a random sample of housing units, and initial nonrespondents will 
receive a replacement questionnaire that mimics the initial questionnaire. The goal of the 
experiment is not to identify specific alternative formats, but rather to learn more about the 
general formats that are preferred in order to fold this information into a longer term research 
program on questionnaire design. 
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The goals of parts (a) and (b) are not clear to the panel. Consequently, it is hard to judge 
whether the experimental designs and sample sizes are suitable and whether the experiments are 
likely to yield useful results. Although the sample size for part (a) may be sufficient to detect 
any economically important change in overall response rates between these two forms of the 
questionnaire, it may not be adequate if one wishes to understand how these changes are related 
to subgroups of the population, size of family, etc. The sample size is also not likely to be 
adequate if one is attempting to relate specific changes in response patterns to specific living 
situations, membership in demographic subgroups, etc. Otherwise, interpretation of any changes 
in response patterns will be limited due to confounding as a result of the several simultaneous 
changes to the questionnaire. As a result, the benefits for questionnaire design for 2020 will be 
reduced. 

With respect to part (b) we are concerned about adequate power because it was unclear 
that 30,000 households would provide a large enough number of alternative addresses to be able 
to determine whether the inclusion of such a question on the census questionnaire would be able 
to substantially affect the need for the coverage follow-up interview or the accuracy of such an 
interview if it appeared to be needed. Therefore, some form of targeting — say of areas with a 
high frequency of seasonal second homes, or of people living in types of group quarters that 
frequently involve duplication — ^would be desirable. Second, it was not clear that this part 
included sufficient provision for gathering follow-up information so as to determine the 
usefulness of the additional question. That is, although the addition of any question on the 
census form has an associated cost of processing and a possible decrease in overall data quality, 
the inclusion of this question could produce higher quality responses as to census residence 
and/or it could also affect the frequency of coverage follow-up interviews or their accuracy. 
Therefore, it is important to include plans in the experimental protocol that would attempt to 
evaluate this tradeoff, since this should be key to making any decisions about the inclusion of 
such a question in the 2020 census questionnaire. 

The race/ethnicity arms of this experiment (part c) involve fine distinctions in question 
wording that are most applicable to specific demographic subgroups. In particular, a major 
emphasis in this section is on Hispanic respondents. Therefore, this experiment would greatly 
benefit from any efforts to target the delivery of the questionnaire to areas with a larger 
percentage of Hispanic residents. In addition, given the increased use of bilingual questionnaires 
in the 2010 census to facilitate response for essentially the same population, it would be useful to 
extend this experiment to examine the impact of such changes on a bilingual version of the 
census questionnaire. 

The Deadline Messaging and Compressed Scheduie Experiment The key question of 
this experiment is whether the rate of mail response could be increased as a result of the use of 
deadline messaging (namely, the use of a notice on the mailing package that the form is required 
to be returned by a specific date) or a compressed mailing schedule or both. In the experiment, 
three sampling strata will be used: high, medium, and low mail response areas. Each of the eight 
study panels will involve 10,000 households. These eight panels are: (1) control, (2) compressed 
mailing schedule panel, (3-5) three deadline messaging panels, and (6-8) three compressed 
schedule combined with deadline messaging panels. The three deadline messaging panels have 
language of varying degrees of sternness related to delays in mailing back the questioimaire. The 
analysis will focus on response rates, speed of response, and item nomesponse rates. Our only 
concern about this experiment is the lack of specification of the statistical power. 
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In summary, as the Census Bureau finalizes its preparations for the 20 1 0 census, the 
panel believes that the Bureau faces tremendous risk if it docs not perform comprehensive 
systems testing — focused on the interfaces between individual system components and, ideally, 
involving some field work component. The quality and utility of 2010 census evaluations will 
also be seriously impaired if census operational systems are not designed to retain procedural 
data for construction of a master trace database. The Census Bureau has proposed four 
experiments to be conducted during the 2010 census, but the panel believes that they suffer from 
design flaws and, significantly, lack connection to potential visions for the 2020 census. The 
panel suggests that three topics that are given little or no weight in the current CPEX plan — 
Internet data collection, use of administrative records in various census processes, and elicitation 
of accurate residence information — ^have greater potential to decrease the cost and increase the 
quality of the 2020 census, and so should be built into the 2010 experimental program. 

We hope that the information and recommendations in this letter are useful to the Census 
Bureau. We would be happy to discuss and explain any of these issues at your convenience. 

Sincerely, 


Lawrence D. Brown, Chair 

Panel on the Design of the 2010 Census 

Program of Evaluations and Experiments (CPEX) 


end: Panel Roster 
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Mr. Chairman and Members of this Senate Subcommittee, I am pleased to be invited to 
provide testimony on the topic, “Lessons Learned: How the New Administration Can 
Achieve An Accurate and Cost-Effective 2010 Census.” My testimony will focus on a 
major lesson learned from prior censuses: the importance of developing strong 
partnerships and community outreach strategies with hard-to-count populations in order 
to reduce the minority undercount in the Census. 

My initial experience with decennial censuses goes back to 1969, when I was appointed 
National Director of the National Urban League’s 1970 Census Project, which was a 
nationwide educational outreach program to reduce the Black undercount in the Census. 
This 1970 Census Project was launched by Whimey M. Young, Jr., the Executive 
Director of the National Urban League at that time. It was the first national partnership 
between the U. S. Census Bureau and a minority organization with over 100 branches 
throughout the nation. The primary purpose of the 1970 Census project, whose slogan 
was “Make Black Count,” was to educate African Americans about the importance of the 
Census and to encourage them to cooperate with the Census. 

Our community outreach project was successful in convincing large segments of the 
African American community to participate in the 1970 Census. However, post-Census 
studies revealed that there was still a sizable undercount of African Americans and other 
minorities in the 1970 Census. We believed that a major reason for the historic 
undercount of minority groups was the failure of the U. S. Census Bureau to adequately 
involve minority representatives in the advance planning and implementation of 
decennial censuses. Therefore, in his testimony to the House Census Oversight 
Committee in September 1970, Mr. Whitney Young recommended that the Census 
Bureau establish on-going Minority Advisory Committees to improve its strategies for 
reducing the Census undercount in hard-to-count communities. 

Indeed, in 1975, under the visionary leadership of Vincent Barabba as Director, the 
Census Bureau formed the first Race and Ethnic Advisory Committees (or “REAC’s” ) 
to assist the Bureau in planning for the 1980 Census. The initial REAC’s comprised four 
minority. groups: African Americans, American Indians, Hispanics and Asians. For the 
2000 Census, a fifth group was added: Native Hawaiians and Pacific Islanders. I served 
as Chair of the African American Advisory Committee in planning for the 1980 Census, 
the 2000 Census and in the initial planning for the 2010 Census. 

Over the years, the Bureau has steadily improved its methods for enumerating the 
American population and for reducing the undercount among minority groups. While 
there is still a differential undercount of minorities, its size has steadily declined. For 
example, while the Bureau estimated that it missed about 1 .5 million (or 8%) of African 
Americans in the 1 970 Census, it failed to count about 1 .0 million (or 1 .84 %) of them in 
the 2000 Census. But the group with the highest undercount rates in decennial censuses- 
regardless of their race or ethnicity— are children under 1 8. 
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One of the most effective strategies the Bureau has used to reduce the minority 
undercount in decennial censuses is to develop strong partnerships with minority groups 
in all phases of census planning and to conduct aggressive education and outreach 
campaigns in hard-to-count communities. Based on my experiences with prior censuses, 

I would like to offer some recommendations to achieve a more accurate count of all 
groups in the 2010 Census. 

First, I think it is very important that Congress provides the Census Bureau with adequate 
resources to undertake the mammoth task of achieving a fair and accurate count of the 
entire American population. President Obama and members of Congress should be 
congratulated for including an additional $1 billion in the President’s Stimulus Bill to 
enhance the Bureau’s enumeration activities in the 20 10 Census. I was especially pleased 
that the Bill stipulates that the Bureau can spend up to $250 million for its Partnership 
Program and outreach efforts to minority communities and hard-to-reach populations. 

Second, because of its comprehensive scope, the Census will directly stimulate this 
economy by hiring over half a million census takers to work throughout the nation. It is 
essential that there is an ethnically and racially diverse workforce from the staff in district 
offices to the enumerators in the neighborhoods and barrios. Members of hard-to-count 
populations should be adequately represented among the census hires at all levels, and 
especially among the new Partnership Specialists. 

Third, one of the remarkable successes of the 2000 Census was the use of paid 
advertising to communicate messages and information about the importance of the 
Census to all groups throughout the nation, regardless of race or ethnicity. The fact that 
minority-owned advertising firms were extensively used to reach their respective groups 
played a large part in reducing the undercount in minority communities in 2000. 

.Fourth, the Bureau should permit the members of its five Race and Ethnic Advisory 
Committee to play a more prominent role in implementing the 2010 Census, such as 
recommending Partnership Specialists and Minority Advertising Firms; distributing 
foreign language census forms; and identifying local sites for training census workers and 
for serving as Assistance Centers to aid the elderly and other individuals to fill out their 
census forms. 

These are a few suggestions that I have to offer to ensure that the 2010 Census will be 
one of the most accurate and equitable enumerations in our history. Thank you for this 
opportunity to share my views on this important issue with this Subcommittee. 
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